Hi! I am trying to run a flow with mapped tasks with a DaskExecutor on my machine, I have a couple of queries : 1. will the overall run time benefit if I run the same flow on Prefect Cloud? 2. Does prefect cloud manage the spawn of a Dask cluster resource ?

Hey @Abhas P , local will be faster than Prefect cloud because each task needs to update its state with the Prefect API at least 3 times. These cause an overhead. Some DaskExecutors allow you to shut down after usage so it’s a setting when you do:

executor = DaskExecutor(
    cluster_class="dask_cloudprovider.FargateCluster",
    cluster_kwargs={
        "image": "prefecthq/prefect:latest",
        "n_workers": 5,
        ...
    },
)

Otherwise you need to use the Prefect Resource Manager