Zach
12/01/2020, 9:38 PMJim Crist-Harif
12/01/2020, 9:40 PMZach
12/01/2020, 9:46 PMKubernetesJobEnvironment
.
For storage I am using prefect.environments.storage.Docker
,
I am not sure what you mean by run_config, I am using the PrefectFlowManager
if that is what you mean, but I don't think it is.Jim Crist-Harif
12/01/2020, 9:47 PMflow.run_config
(see https://docs.prefect.io/orchestration/flow_config/overview.html) - these will replace environments (after a long deprecation cycle).Docker
storage building. This is less a question about how to configure prefect, and more a question about how to speedup docker storage builds. A few tips:
• If you have lots of custom dependencies, you might build a custom base image used by all your flows, and set this as base_image
. Then your flow build would only need to add the flow source to the image before registering.
• Alternatively you might use a different storage type (S3
, Github
, ...) for hosting your flow source, and configure a static image on your flow separately. To do that using the legacy environment-based system you'd set flow.environment.metadata["image"] = "your-image-name"
). If you're using a KubernetesRun
run-config, this would just be the image
parameter to that object.