Hi Is it possible to restrict number of parallel map tasks w

Question

Hi Is it possible to restrict number of parallel map tasks with DaskExecutor

Kevin Kho · Answer

It would be by limiting the number of threads processes when creating the Executor We also do have <https docs prefect io orchestration flow runs concurrency limits html task run limits|task run> concurrency limiting on Prefect Cloud tho

Vadym Dytyniak · Answer

But Prefect will use all available threads processes in the Dask cluster

Vadym Dytyniak · Answer

I think only second approach will work for me

Kevin Kho · Answer

Oh you already have a cluster you just connect to

Vadym Dytyniak · Answer

correct