https://prefect.io logo
j

Jeff Brainerd

11/06/2020, 2:08 AM
Hi P team, a question about current best practices running Dask on Fargate…
We are currently using
LocalEnvironment
with a
DaskExecutor
. We are managing a semi-persistent dask cluster ourselves (i.e. the same cluster runs multiple flow runs). We would like to shift to a scheme that creates an ephemeral dask cluster per flow run. Is the current best practice to use
LocalEnvironment
with
Copy code
DaskExecutor(
    cluster_class="dask_cloudprovider.FargateCluster", ...
)
? Should I stay away from
DaskCloudProviderEnvironment
? Thanks for any info! 🙏
d

Dylan

11/06/2020, 3:28 PM
Hi @Jeff Brainerd, That’s our recommended path forward 👍
j

Jeff Brainerd

11/06/2020, 3:45 PM
Thanks @Dylan!
d

Dylan

11/06/2020, 3:45 PM
You got it Jeff!