Hey! What would you say is the best way of running prefect in production with minimal flow / task startup time?
Philipp Eisen
01/21/2022, 2:31 PM
Standing Dask cluster and k8s agent with a very small python image?
a
Anna Geller
01/21/2022, 2:38 PM
you can probably get the lowest latency with a local agent or when using Kubernetes/Docker agent with worker nodes having the images locally to the node
Anna Geller
01/21/2022, 2:40 PM
on Kubernetes, you can specify imagePullPolicy set to
IfNotPresent
so that the image is pulled only if it is not already present locally
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.