Joseph Wibowo
11/14/2025, 9:33 PMMarvin
11/14/2025, 9:33 PMMarvin
11/14/2025, 9:36 PMJoseph Wibowo
11/14/2025, 9:39 PMprefect deploy we will use one docker image for all deployments. If we had hundreds of deployments though - how long would it takeJoseph Wibowo
11/14/2025, 9:41 PMprefect deploy we will use one docker image for all deployments. If we had hundreds of deployments though - how long would it takeMarvin
11/14/2025, 9:41 PMMarvin
11/14/2025, 9:42 PMprefect deploy won’t spend time building/pushing images. In that case, deploy time is mostly:
- importing each flow
- validating config
- registering the deployment with the API
Rule of thumb (Prefect 3.x)
- Per deployment: typically 1–5 seconds if imports are light and the network/API is healthy
- 100 deployments: roughly 2–10 minutes, linearly scaling with count
- If imports are heavy (big libraries, top-level work at import): 5–15 seconds each → 8–25 minutes for 100
For “hundreds,” you can estimate linearly. Example:
- 300 deployments at ~3s each → ~15 minutes
- 300 deployments at ~8s each → ~40 minutes
How to get a quick, accurate estimate in your environment
- Time a subset and extrapolate. For example, deploy 10 by pattern:
prefect deploy --name 'my-flow-prefix/*'
If that takes ~30s, you’re ~3s/deployment → 100 ≈ 5 minutes. You can deploy all at once with:
prefect deploy --all
Tips to keep it fast
- Keep flow imports lightweight (avoid heavy work at module import time).
- Consolidate deployments in one project and deploy in a single command to minimize overhead.
- If you do parallelize multiple deploy commands, be mindful of API load; it’s usually best to keep to a small concurrency.
References
- prefect.yaml and multi-deployment projects
- Deploy via Python (flow.deploy)
- Case study on cutting hundreds-of-deployments time by eliminating per-deployment builds: enterprise case study
If you can share roughly how heavy your imports are (or a timing from 10 deployments), I can give a tighter estimate for “hundreds.”Joseph Wibowo
11/14/2025, 9:46 PMMarvin
11/14/2025, 9:46 PMMarvin
11/14/2025, 9:47 PMprefect deploy --name 'my_flow/*'
- A single deployment:
prefect deploy --name 'my_flow/prod'
- Wildcards work (e.g., *-flow/*, */staging-*)
- By entrypoint (file:function) to target just that flow’s deployments:
prefect deploy ./flows/my_flow.py:my_flow
You can combine these if you want to be extra specific (e.g., entrypoint + a name pattern).
CI tip for “only deploy what changed”
- Detect changed flows and deploy only those:
for f in $(git diff --name-only origin/main...HEAD | grep '^flows/.*\.py$'); do
flow_func=$(basename "$f" .py)
prefect deploy "$f:$flow_func"
done
Or, if your deployments are named flow/deployment, map changed flow to a name pattern:
prefect deploy --name 'my_flow/*'
Docs
- prefect.yaml and multi-deployment projects
- CLI help shows examples for the --name pattern; run:
prefect deploy --helpJoseph Wibowo
11/14/2025, 10:00 PMprefect.yaml files? Can you specify each prefect.yaml?Marvin
11/14/2025, 10:01 PMMarvin
11/14/2025, 10:05 PM--prefect-file, so multiple prefect.yaml files are fine—just run one command per file.
Examples
- Deploy all deployments from a specific file:
prefect deploy --prefect-file services/payments/prefect.yaml --all
- Deploy only certain deployments from that file (by name pattern):
prefect deploy --prefect-file services/payments/prefect.yaml --name 'payments-flow/*'
- Deploy a single flow by entrypoint (within that file’s project):
prefect deploy --prefect-file services/payments/prefect.yaml ./flows/payments.py:payments_flow
Notes
- You cannot pass multiple --prefect-file values in a single command. Run one prefect deploy per file.
- Filtering with --name applies to the deployments defined in the file you point to.
- --all deploys all deployments defined in that specific file.
- Run the command from your project root so entrypoints resolve correctly (or use absolute paths).
CI tip (deploy only changed project files)
for f in $(git diff --name-only origin/main...HEAD | grep 'prefect\.yaml$'); do
prefect deploy --prefect-file "$f" --all --no-prompt
done
Docs
- prefect.yaml and multi-deployment projects