Ed Hartley
06/09/2026, 8:29 AMMarvin
06/09/2026, 8:29 AMMarvin
06/09/2026, 8:31 AMpip_install_requirements from pull.
- If needed, disable auto-install behavior with:
PREFECT_RUNNER_AUTO_INSTALL_DEPENDENCIES=false
2. Use a persistent environment
- If your worker runs in a long-lived environment and dependencies are already installed, don't reinstall them for every run.
3. Avoid pulling/installing on every execution
- If you're pulling code from Git, consider using storage options that cache between runs, like GitRepository(..., pull_interval=...), so code isn't refreshed every single time.
Why it's slow
pip_install_requirements runs during deployment/workspace preparation, so every flow run may:
- read requirements.txt
- resolve deps
- download packages
- install them
That can be especially slow with large packages like pandas, torch, db drivers, etc.
Recommended pattern
Instead of this in `prefect.yaml`:
pull:
- prefect.deployments.steps.git_clone:
repository: <https://github.com/my-org/my-repo.git>
- prefect.deployments.steps.pip_install_requirements:
requirements_file: requirements.txt
Prefer a prebuilt image where your Dockerfile does:
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . /app
WORKDIR /app
Then your deployment just points at that image.
If you must keep runtime installs
A few things can help:
- keep requirements.txt small
- pin versions to reduce resolver work
- use a pip cache / persistent volume if your infra supports it
- avoid reinstalling unchanged local packages
- use uv-based/project installs if that's already part of your environment
Prefect docs
- prefect.yaml deployment steps
- store flow code / deployment storage
If you want, I can help you optimize this for your specific setup — e.g. Docker worker, Kubernetes worker, process worker, or Git-based deployment.