<@ULVA73B9P> if i use `flow.from_source` deploymen...
# ask-marvin
a
@Marvin if i use
flow.from_source
deployment, how do i ensure uv sync and uv run is used to run the flow
m
thought for 365.4 seconds
Short answer: wrap Prefect’s run command with uv in your worker’s job template (or per-deployment job variables) so the worker does
uv sync && uv run {{ command }}
. That ensures dependencies are synced at run time and the flow executes under uv. Below are concrete options. Assumptions: Prefect 3.x and you’re using
flow.from_source(...).deploy(...)
. If you’re on a different worker type than “process” or “docker”, tell me which one. General approach - Make sure
uv
is installed where your worker runs (on the host for process workers; in the image for Docker/K8s). - Configure the worker’s command to: 1) run
uv sync
in the project directory (pulled by Prefect’s source step) 2) run the actual flow command with
uv run
- Do this once in the work pool’s base job template (recommended) or per-deployment via
job_variables.command
. Why this works: Prefect injects the actual flow execution command into the template as
{{ command }}
. Wrapping it as
uv run {{ command }}
preserves all the parameters Prefect passes while putting uv in charge. Process worker (local) 1) Create a work pool with a custom base job template that wraps the command:
Copy code
{
  "variables": {
    "type": "object",
    "properties": {
      "env": {
        "type": "object",
        "default": {}
      },
      "labels": {
        "type": "object",
        "default": {}
      },
      "name": {
        "type": "string",
        "default": null
      },
      "working_dir": {
        "type": "string",
        "default": null
      },
      "command": {
        "type": "string",
        "default": null
      }
    }
  },
  "job_configuration": {
    "env": "{{ env }}",
    "labels": "{{ labels }}",
    "name": "{{ name }}",
    "working_dir": "{{ working_dir }}",
    "stream_output": true,
    "command": "bash -lc 'uv sync && uv run {{ command }}'"
  }
}
Then:
Copy code
prefect work-pool create "process-uv" --type process --base-job-template ./base-job-template.json
(We verified the CLI:
prefect work-pool create --help
supports
--base-job-template
and
--type
.) 2) Deploy with
from_source
targeting this pool:
Copy code
from prefect import flow

f = flow.from_source(
    source="<https://github.com/your-org/your-repo>",
    entrypoint="flows/etl.py:etl",
    work_pool_name="process-uv",
)

f.deploy(name="etl-uv")
Per-deployment override (optional): if you don’t want to touch the pool template, you can set the command when deploying:
Copy code
f.deploy(
    name="etl-uv",
    work_pool_name="process-uv",
    job_variables={"command": "bash -lc 'uv sync && uv run {{ command }}'"},
)
Docker worker - Ensure your image contains
uv
and your project’s
pyproject.toml
. - Use a base job template that runs uv before Prefect’s command (typical Docker/K8s templates use array syntax for command):
Copy code
{
  "variables": {
    "type": "object",
    "properties": {
      "image": {
        "type": "string"
      },
      "env": {
        "type": "object",
        "default": {}
      },
      "labels": {
        "type": "object",
        "default": {}
      },
      "name": {
        "type": "string",
        "default": null
      },
      "working_dir": {
        "type": "string",
        "default": null
      },
      "command": {
        "type": "string",
        "default": null
      }
    },
    "required": [
      "image"
    ]
  },
  "job_configuration": {
    "image": "{{ image }}",
    "env": "{{ env }}",
    "labels": "{{ labels }}",
    "name": "{{ name }}",
    "working_dir": "{{ working_dir }}",
    "command": [
      "bash",
      "-lc",
      "uv sync && uv run {{ command }}"
    ]
  }
}
Then:
Copy code
prefect work-pool create "docker-uv" --type docker --base-job-template ./docker-template.json
Notes and gotchas - uv must be on PATH where the worker runs. For process workers, install uv on the machine running `prefect worker start`; for Docker, bake it into the image. - The working directory: Prefect checks out your source before running the command, and sets the working directory appropriately; including
"working_dir": "{{ working_dir }}"
ensures
uv sync
runs in the repo root.
- Speed:
uv sync
is fast, but if you’re doing this on every run, consider enabling caching (e.g., a persistent
.venv
directory) so uv can reuse it between runs. - Windows workers: replace
bash -lc '…'
with a PowerShell equivalent, or invoke a shell script in your repo instead. A simple one-liner wrapper script like
scripts/run_with_uv.sh
can save you from quoting headaches: - `scripts/run_with_uv.sh`:
Copy code
#!/usr/bin/env bash
    set -euo pipefail
    uv sync
    uv run "$@"
- Template command:
Copy code
"command": "bash -lc './scripts/run_with_uv.sh {{ command }}'"
If you tell me which worker type you’re using, I’ll tailor the exact base job template and the deploy snippet for it. References (because trust is a thing we still cling to): - Deployments (Prefect 3) - Workers and Work Pools - Managing Work Pools and Job Templates - Blog: What data professionals need to know about uv
a
@Marvin i am using the default serverless from prefect
m
thought for 79.7 seconds
Ah, Serverless. Convenient, but not exactly pliable. In Prefect Cloud Serverless, the runtime image and command are managed by Prefect. You can’t change the entrypoint, inject pre-run steps, or wrap the execution in uv. So: - You cannot run uv sync before your flow. - You cannot force the flow to execute under uv run. - The Serverless job template is fixed and not user-editable. Workable options - Keep Serverless, but use uv to produce pinned requirements for install: - Use uv locally to export/compile your dependencies into a requirements.txt (from pyproject.toml), commit it, and let Serverless install with pip at runtime. - Example:
Copy code
uv pip compile pyproject.toml -o requirements.txt
- You get uv’s resolver/lock benefits; you just won’t execute under uv run. - If you need true uv sync + uv run at runtime, switch to a customizable worker: - Process worker (local machine) or Docker/Kubernetes work pools let you override the command/entrypoint. There you can set the command to:
Copy code
bash -lc 'uv sync && uv run {{ command }}'
- I can give you a ready-to-use base job template for the worker you choose. Docs, for when optimism strikes: - Deployments: Deployments (Prefect 3) - Workers and Work Pools: Workers and Work Pools - Managing Work Pools / Job Templates: Manage Work Pools and Job Templates If you want to switch from Serverless, tell me which worker type (process, docker, k8s) and I’ll hand you the exact template and deploy snippet. Otherwise, I can show the uv → requirements.txt workflow to keep Serverless happy.
n
hm there’s some nuance here marvin is missing, afk but i should make an issue for this if we don’t have one there’s a sequencing issue, lemme see if i can find the similar thread about this