Jordan Fisher
12/01/2025, 2:13 PMMarvin
12/01/2025, 2:13 PMMarvin
12/01/2025, 2:18 PM/opt/prefect/flows. The worker will run every flow in that same directory.
- Alternatively, add a pull step to set the working directory:
pull:
- prefect.deployments.steps.set_working_directory:
directory: /opt/prefect/flows
Note: this only helps if that path is actually persistent on the machine/volume where the worker runs.
- Pin your deployment to a commit SHA in the git pull step:
pull:
- prefect.deployments.steps.set_working_directory:
directory: /opt/prefect/flows
- prefect.deployments.steps.git_clone:
repository: <https://github.com/owner/repo.git>
commit_sha: "<commit-sha>"
With a persistent working dir:
- First run clones and checks out the commit.
- Subsequent runs detect the repo is already at that commit and no-op locally (no remote call), so runs won’t crash if the git host is down.
Notes and caveats
- If you pull by branch (not commit), the step will attempt a git pull and will fail if the provider is down. For maximum resilience, pin to a commit SHA.
- The built-in step does shallow clones by default and reuses existing repos in the working directory. It validates by commit SHA when provided.
- If you run workers in ephemeral environments (e.g., new containers/pods each run) you’ll need a persistent volume mount for the working directory to get the benefits.
If you need even stronger offline behavior
- You can create a small custom pull step that:
- Checks for .git and whether the requested commit exists locally.
- If it does, returns early without any git network commands.
- If not, it fetches only as needed.
This is a straightforward extension because pull steps are just Python callables.
References
- Process worker working directory (source): prefect/workers/process.py
- Git pull step implementation (source): prefect/deployments/steps/pull.py
- Deployments and pull steps overview: Deployments docs
A couple quick questions to make sure this will work for you:
- Are you able to set a persistent working directory on the machine (or mount a persistent volume if containerized)?
- Are your deployments pinned to commit SHAs, or are you currently pulling by branch?