Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Registering my flows takes a (relatively) really long time, is there anything I can do to speed it up

Hi Zach, without knowing more about what you're doing it's hard to answer this. What kind of flow storage/environment/run_config are you using?

I am using the `KubernetesJobEnvironment`.
For storage I am using `prefect.environments.storage.Docker`,
I am not sure what you mean by run_config, I am using the `PrefectFlowManager` if that is what you mean, but I don't think it is.

We're moving to a new configuration system for flow-runs, configured by `flow.run_config` (see <https://docs.prefect.io/orchestration/flow_config/overview.html>) - these will replace environments (after a long deprecation cycle).

The main slowdown for you would then be `Docker` storage building. This is less a question about how to configure prefect, and more a question about how to speedup docker storage builds. A few tips:

• If you have lots of custom dependencies, you might build a custom base image used by all your flows, and set this as `base_image`. Then your flow build would only need to add the flow source to the image before registering.
• Alternatively you might use a different storage type (`S3` , `Github`, ...) for hosting your flow source, and configure a static image on your flow separately. To do that using the legacy environment-based system you'd set `flow.environment.metadata["image"] = "your-image-name"`). If you're using a `KubernetesRun` run-config, this would just be the `image` parameter to that object.