In prod what is the most typical Dask cluster set up 1

Question

In prod what is the most typical Dask cluster set up 1 2 If existing cluster what is the most popular cluster manager Yarn K8s Thanks

Kevin Kho · Answer

Temporary on k8s fargate or hosted services like Coiled and Saturn Cloud because you can install packages you need easily per flow at runtime

Kevin Kho · Answer

But really it s upto you I think Dask recommends long running I don t know why maybe long running access to the dashboard

Shiyu Gan · Answer

Can you link me instruction from prefect or dask on temporary on k8s fargate

Kevin Kho · Answer

<https medium com slateco blog prefect x kubernetes x ephemeral dask power without responsibility 6e10b4f2fe40|This> has an example of using the KubeCluster