Thread
#prefect-community
    Jeff Brainerd

    Jeff Brainerd

    1 year ago
    Hi 😛refect: team, a question about current best practices running Dask on Fargate…
    We are currently using
    LocalEnvironment
    with a
    DaskExecutor
    . We are managing a semi-persistent dask cluster ourselves (i.e. the same cluster runs multiple flow runs). We would like to shift to a scheme that creates an ephemeral dask cluster per flow run. Is the current best practice to use
    LocalEnvironment
    with
    DaskExecutor(
        cluster_class="dask_cloudprovider.FargateCluster", ...
    )
    ? Should I stay away from
    DaskCloudProviderEnvironment
    ? Thanks for any info! 🙏
    Dylan

    Dylan

    1 year ago
    Hi @Jeff Brainerd, That’s our recommended path forward 👍
    Jeff Brainerd

    Jeff Brainerd

    1 year ago
    Thanks @Dylan!
    Dylan

    Dylan

    1 year ago
    You got it Jeff!