Josh Connors
09/10/2025, 10:03 AMThe existing repository at /opt/prefect/{REPOSITORY_NAME} does not match the configured repository <https://x-access-token>:{MY_GITHUB_TOKEN}@github.com/{MY_GITHUB_USERNAME}/{REPOSITORY_NAME}.git
Marvin
09/10/2025, 10:03 AMMarvin
09/10/2025, 10:10 AM/opt/prefect/{REPOSITORY_NAME}
.
- There’s already a Git repo at that exact path, but its configured remote URL is different from the one your deployment is trying to use (often because:
- you previously cloned a different owner’s repo with the same name, or
- the remote URL includes a different token/format now, so the strings don’t match).
Common scenarios
- Two different GitHub repos that share the same name (e.g., org/repo and personal/repo) run on the same worker and collide at /opt/prefect/repo
.
- You rotated tokens or changed clone URLs and now the previously cached repo’s remote URL doesn’t match the new one.
Quick ways to resolve
Option A — Clean the directory before clone (recommended)
- Add a shell step to remove the directory before git_clone
in your deployment’s pull steps:
from prefect import flow
@flow
def my_flow():
...
my_flow.from_source(
source="<https://github.com/{MY_GITHUB_USERNAME}/{REPOSITORY_NAME}>",
entrypoint="path/to/flow.py:my_flow",
).deploy(
name="prod",
work_pool_name="default-work-pool",
pull=[
{"prefect.deployments.steps.utility.run_shell_script": {
"script": "rm -rf /opt/prefect/{REPOSITORY_NAME} || true"
}},
{"prefect.deployments.steps.pull.git_clone": {
"repository": "<https://github.com/{MY_GITHUB_USERNAME}/{REPOSITORY_NAME}.git>",
"access_token": "${{ PREFECT_GH_TOKEN }}", # or use a credentials block
"branch": "main"
}},
{"prefect.deployments.steps.pull.set_working_directory": {
"directory": "/opt/prefect/{REPOSITORY_NAME}"
}},
],
)
Option B — Update the existing repo’s remote to match your configured URL
- If it’s truly the same repo and only the token or URL format changed, you can set the remote before the clone step:
{
"prefect.deployments.steps.utility.run_shell_script": {
"script": "git -C /opt/prefect/{REPOSITORY_NAME} remote set-url origin <https://x-access-token:${PREFECT_GH_TOKEN}@github.com/{MY_GITHUB_USERNAME}/{REPOSITORY_NAME}.git> || true"
}
}
Option C — Manually delete the cached directory on the worker
- If you control the machine/container where the worker runs, remove /opt/prefect/{REPOSITORY_NAME}
and re-run. This is the same as Option A but done outside the deployment.
Notes and next steps
- In Prefect 3.x, deployments from Git use pull steps under the hood. The clone step is `prefect.deployments.steps.pull.git_clone`; docs: API reference.
- If you’re running multiple repos with the same name on the same worker, consider always cleaning before clone to avoid collisions.
- Whether this is running “in Prefect Cloud” depends on your work pool:
- If you’re using a self-hosted worker (Process/Docker/Kubernetes), the repo lives on that worker, not in Prefect Cloud.
- If you’re using the Serverless work pool, the environment is ephemeral, and you typically shouldn’t see leftovers—if you do, share details and we’ll dig in.
Could you share:
- Your deploy(...)
call (redact tokens), especially the pull
steps
- Which work pool type you’re using (Process/Docker/Kubernetes/Serverless)
- Whether you have multiple repos with the same {REPOSITORY_NAME}
on the same worker
Helpful links
- Deploy from Git (Prefect 3): How to: Deploy from Git
- Pull steps API (includes git_clone
, set_working_directory
, shell script): Pull steps APIJosh Connors
09/10/2025, 10:22 AMuv
the package manager to handle Python dependencies. I was not able to locate documentation that shows how to install dependencies onto the worker via uv. Can you show me how that is possible within the prefect.yaml
file?Marvin
09/10/2025, 10:22 AMMarvin
09/10/2025, 10:26 AMname: my-project
prefect-version: ">=3.0.0"
deployments:
- name: prod
entrypoint: path/to/flow.py:my_flow
work_pool:
name: default-work-pool
pull:
- prefect.deployments.steps.pull.git_clone:
repository: <https://github.com/{MY_GITHUB_USERNAME}/{REPOSITORY_NAME}.git>
access_token: ${{ PREFECT_GH_TOKEN }}
branch: main
- prefect.deployments.steps.utility.run_shell_script:
# Install into the worker's active interpreter
directory: /opt/prefect/{REPOSITORY_NAME}
script: |
# If you use requirements.txt
uv pip install -r requirements.txt
# Or, if you have a package/pyproject and want to install your project
# uv pip install -e .
- prefect.deployments.steps.pull.set_working_directory:
directory: /opt/prefect/{REPOSITORY_NAME}
Important notes:
- uv must already be installed on the worker host/container. If not, add a step to install it first (e.g., pip install uv
or include it in your base image).
- Installing into the worker’s interpreter is generally the simplest with the Process work pool. If you prefer an isolated, per-run virtual environment (e.g., uv sync
to .venv), the Process worker won’t automatically use that venv to run your flow. You’d either:
- Switch to Docker/Kubernetes and bake dependencies into the image using uv, or
- Heavily customize the process launch (not typical) so the flow runs via the venv’s interpreter.
- If multiple deployments share a worker and install different versions of packages at runtime, you may get conflicts. For full isolation, Docker/Kubernetes images with uv-installed dependencies are preferred.
If you’d like a Docker-based example, here’s a minimal pattern for a Docker work pool image:
# Dockerfile
FROM python:3.11-slim
RUN pip install --no-cache-dir uv prefect
WORKDIR /opt/prefect/app
COPY pyproject.toml uv.lock ./ # or requirements.txt
# With pyproject:
# RUN uv pip install -e . # installs your project and deps into system interpreter
# With requirements:
# RUN uv pip install -r requirements.txt
Then point your deployment at that image via a Docker work pool.
Docs you may find helpful:
- prefect.yaml reference: prefect.yaml guide
- Pull steps (includes run_shell_script
): utility steps API
- Git clone pull step: pull steps API
A couple of quick questions to tailor this further:
- Are you using a requirements.txt or a pyproject.toml/uv.lock workflow?
- Do you want per-run isolation, or is installing into the worker’s environment acceptable?
- Are you on the Process work pool, or do you have Docker/Kubernetes available?Josh Connors
09/10/2025, 10:31 AMMarvin
09/10/2025, 10:31 AMMarvin
09/10/2025, 10:32 AMname: my-project
prefect-version: ">=3.0.0"
deployments:
- name: prod
entrypoint: path/to/flow.py:my_flow
work_pool:
name: default-work-pool
pull:
# 1) Pull source
- prefect.deployments.steps.pull.git_clone:
repository: <https://github.com/{MY_GITHUB_USERNAME}/{REPOSITORY_NAME}.git>
access_token: ${{ PREFECT_GH_TOKEN }}
branch: main
# 2) Ensure uv is installed on the worker, then install deps + your project
- prefect.deployments.steps.utility.run_shell_script:
directory: /opt/prefect/{REPOSITORY_NAME}
script: |
# Ensure uv is available in the worker's current interpreter
command -v uv >/dev/null 2>&1 || python -m pip install --upgrade uv
# Install your project and dependencies from pyproject.toml / uv.lock
# This installs into the active interpreter used by the Process worker
uv pip install -e .
# 3) Set working directory for the run
- prefect.deployments.steps.pull.set_working_directory:
directory: /opt/prefect/{REPOSITORY_NAME}
Notes and options:
- If you use extras, install with:
- uv pip install -e ".[prod]"
(or whatever extra you use)
- If you also want dev deps, add the extra or include them in your pyproject.
- Because you’re on the Process work pool, installs persist in the worker’s environment and subsequent runs will be fast and idempotent.
- Make sure the worker’s Python environment is writable by the user running the worker. If it’s system Python, consider running the worker in a venv you control.
Docs:
- prefect.yaml guide: prefect.yaml
- Pull steps API (includes git_clone
, set_working_directory
, run_shell_script
): pull steps and utility steps
If you want me to adapt this to your exact repo layout (e.g., different entrypoint, a monorepo path, or extras), share the paths/extras and I’ll tailor the script.Josh Connors
09/10/2025, 10:38 AMuv pip install -e .
command; is it better to use uv sync
instead?Marvin
09/10/2025, 10:38 AMMarvin
09/10/2025, 10:39 AMuv pip install -e .
in your pull steps for a Process work pool.
Why not uv sync
here?
- uv sync
creates/updates a dedicated virtual environment for the project (typically .venv) and applies your uv.lock.
- The Prefect Process worker does not automatically switch to that virtualenv to run your flow. It uses the Python interpreter the worker was started with.
- So running uv sync
in a pull step won’t change the interpreter your flow uses, and you’ll still end up running with the worker’s environment.
Recommended for your setup (Process pool, OK installing into worker environment):
- Keep using uv pip install -e .
(or uv pip install -e ".[prod]"
) in a pull step. This installs your project and dependencies into the worker’s active interpreter before the flow runs.
If you really want uv sync
with lockfile enforcement:
- Option A: Pre-provision the worker environment and launch the worker from that interpreter
- Create a venv using uv sync
outside of Prefect (on the worker host), then start the worker from that venv’s Python so all runs use that environment.
- Option B: Use Docker/Kubernetes and bake uv sync
into the image at build time, then run your deployment on that image.
For reference, the pull step approach you’ve been using is the right fit for Process workers:
pull:
- prefect.deployments.steps.pull.git_clone:
repository: <https://github.com/{MY_GITHUB_USERNAME}/{REPOSITORY_NAME}.git>
access_token: ${{ PREFECT_GH_TOKEN }}
branch: main
- prefect.deployments.steps.utility.run_shell_script:
directory: /opt/prefect/{REPOSITORY_NAME}
script: |
command -v uv >/dev/null 2>&1 || python -m pip install --upgrade uv
uv pip install -e ".[prod]" # or just -e . if you don't use extras
- prefect.deployments.steps.pull.set_working_directory:
directory: /opt/prefect/{REPOSITORY_NAME}
Docs:
- Pull steps (git_clone, set_working_directory, run_shell_script): pull steps and utility stepsJosh Connors
09/10/2025, 10:45 AMFileNotFoundError: [Errno 2] No such file or directory: 'command'
Marvin
09/10/2025, 10:45 AMMarvin
09/10/2025, 10:46 AMcommand -v uv ...
failed because command
is a shell builtin, not an executable. The step likely executed your script without an interactive shell, so it tried to run a program literally named command
, which doesn’t exist.
Two easy fixes
Option A (simplest, recommended): just install uv unconditionally
- This avoids shell builtins and works reliably with the Process work pool.
pull:
- prefect.deployments.steps.pull.git_clone:
repository: <https://github.com/{MY_GITHUB_USERNAME}/{REPOSITORY_NAME}.git>
access_token: ${{ PREFECT_GH_TOKEN }}
branch: main
- prefect.deployments.steps.utility.run_shell_script:
directory: /opt/prefect/{REPOSITORY_NAME}
script: |
python -m pip install -q --upgrade uv
uv pip install -e .
- prefect.deployments.steps.pull.set_working_directory:
directory: /opt/prefect/{REPOSITORY_NAME}
Option B (only if you really want a conditional check)
- Explicitly invoke a shell yourself to access builtins/operators:
- prefect.deployments.steps.utility.run_shell_script:
directory: /opt/prefect/{REPOSITORY_NAME}
script: |
sh -lc '
uv --version >/dev/null 2>&1 || python -m pip install -q --upgrade uv
uv pip install -e .
'
Notes
- With the Process work pool, this installs into the worker’s active interpreter, which matches your intended setup.
- Idempotency: repeated runs will be quick; uv pip install -e .
will no-op if already satisfied.
API reference for the step you’re using: run_shell_scriptJosh Connors
09/10/2025, 10:53 AM`run_shell_script` failed with error code 1: WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: <https://pip.pypa.io/warnings/venv>. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
Error 2:
Using Python 3.12.11 environment at: /usr/local
× No solution found when resolving dependencies:
╰─:arrow_forward: Because the current Python version (3.12.11) does not satisfy
Python>=3.13
Marvin
09/10/2025, 10:53 AMMarvin
09/10/2025, 10:53 AMpyproject.toml
declares requires-python >= 3.13
, but the Process worker is running Python 3.12.11. uv
can’t resolve a solution, so the step exits with code 1.
Fix options
- Best: Run the Process worker with Python 3.13, then keep using uv pip install -e .
in your pull steps.
- Alternative: Relax your project’s requires-python
in pyproject.toml
(only if acceptable).
- Or: Switch to a Docker/Kubernetes work pool and use an image with Python 3.13.
How to run the Process worker with Python 3.13
- Create and use a Python 3.13 environment for the worker, e.g. with a venv:
# Install Python 3.13 via your preferred method (e.g., pyenv, OS packages)
python3.13 -m venv /opt/prefect-venvs/py313
source /opt/prefect-venvs/py313/bin/activate
pip install --upgrade pip uv prefect
# Start your worker from this environment
prefect worker start --pool default-work-pool
- From now on, deployments that run on this worker will use Python 3.13.
Update your pull step to suppress the root warning and install with uv
- You can either set an env var or pass pip’s flag. Here are two options:
Option A (env var)
- prefect.deployments.steps.utility.run_shell_script:
directory: /opt/prefect/{REPOSITORY_NAME}
env:
PIP_ROOT_USER_ACTION: ignore
script: |
python -m pip install -q --upgrade uv
uv pip install -e ".[prod]" # or -e .
Option B (pip flag)
- prefect.deployments.steps.utility.run_shell_script:
directory: /opt/prefect/{REPOSITORY_NAME}
script: |
python -m pip install -q --root-user-action=ignore --upgrade uv
uv pip install -e ".[prod]"
Notes
- The pip “root” warning is benign; the version mismatch is why the step failed.
- With a Process pool, installs persist in the worker’s environment and subsequent runs will be fast.
- If you need strict per-run isolation, consider a Docker work pool and bake Python 3.13 + uv into the image.
References
- Pull steps API (includes run_shell_script
): utility steps and pull steps
- Pip root warning context: pip venv warning
If upgrading the worker to Python 3.13 is an option for you, that will resolve the uv
solver error. Want me to suggest a Dockerfile or pyenv commands for installing Python 3.13 on your host?Josh Connors
09/10/2025, 10:57 AMMarvin
09/10/2025, 10:57 AMMarvin
09/10/2025, 10:59 AMpython_version
(or similar) setting in prefect.yaml.
- Process work pool: the Python version is whatever interpreter the worker process is running under. To change it, start your worker from a Python 3.13 environment/venv.
- Example flow (run on the worker host):
# Create a Python 3.13 environment for the worker
python3.13 -m venv /opt/prefect-venvs/py313
source /opt/prefect-venvs/py313/bin/activate
pip install --upgrade pip uv prefect
# Start your worker from this environment (so all runs use Python 3.13)
# (Run your usual worker start command here)
- Docker/Kubernetes work pools: you control the Python version via the image. In prefect.yaml, point the deployment at an image that already has the Python you need. For example, with a Docker work pool:
deployments:
- name: prod
entrypoint: path/to/flow.py:my_flow
work_pool:
name: docker-pool
job_variables:
image: python:3.13-slim # or your custom image with uv/prefect preinstalled
Given your setup (Process work pool), the fix for your uv solver error is to run the worker with Python 3.13; prefect.yaml cannot change it.
References
- prefect.yaml guide: prefect.yaml
- Docker deployment guidance: Docker deployments