Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

In prod, what is the most typical Dask cluster set up?
1. <https://docs.prefect.io/orchestration/flow_config/executors.html#using-a-temporary-cluster|Temporary cluster>?
2. <https://docs.prefect.io/orchestration/flow_config/executors.html#connecting-to-an-existing-cluster|Existing cluster>?
If existing cluster, what is the most popular cluster manager? Yarn, K8s?
Thanks!

Temporary on k8s/fargate or hosted services like Coiled and Saturn Cloud because you can install packages you need easily per flow at runtime

But really it’s upto you. I think Dask recommends long running. I don’t know why, maybe long-running access to the dashboard?

Can you link me instruction (from prefect or dask) on temporary on k8s/fargate?

<https://medium.com/slateco-blog/prefect-x-kubernetes-x-ephemeral-dask-power-without-responsibility-6e10b4f2fe40|This> has an example of using the KubeCluster