Hello everyone, is there such a way to use Dask Temporary Cluster, created by Prefect, inside a task in order to execute any workload on that Dask cluster?
k
Kyle Moon-Wright
02/02/2021, 1:40 AM
Hey @Giovanni Giacco,
Check out this idiom on creating a temporary Dask Cluster. In the example, you’ll notice the usage of a Prefect resource_manager cluster to submit specific tasks to.