Short answer: yes. In Prefect 3.x, pulling code at run start adds “cold start” time and the cost grows with the amount of data/files you need to fetch.
What actually happens depends on how you store your code:
- Git repo via flow.from_source(...): the worker clones the repo into a fresh working directory at run start (via the steps in prefect.deployments.steps.pull). There’s no built‑in cross‑run cache, so new pods/containers reclone. Large repos and submodules increase startup time. Prefect’s git clone step does not expose shallow/sparse clone flags today.
- Object storage (S3/GCS/etc.): the worker downloads your packaged project/files at run start. Time scales with bytes transferred and number of objects (many small files can be slower than a single archive).
Ways to reduce startup time:
- Bake your code into the runtime image (Docker/Kubernetes) so there’s no runtime fetch.
- Reuse a persistent working directory/volume so subsequent runs don’t reclone/redownload (you manage the lifecycle; Prefect won’t do this automatically).
- Scope what you ship:
- Keep your flow entrypoint in a small subdirectory and avoid large, unrelated files.
- For object storage, ship a single archive instead of many small files.
- If you must use Git, pin to a commit SHA for deterministic pulls; if you control the environment, you can implement your own shallow/sparse clone before the flow starts and point Prefect at that location.
Docs:
-
Store flow code (3.x)
-
Code pull steps API
If you can share how you’re storing code (Git vs S3/GCS) and where your flows run (local process, Docker, K8s), I can suggest the fastest setup for your case.