Maybe this is a dask questions I ll admit I have not properl

Question

Maybe this is a dask questions I ll admit I have not properly investigated this much today s job but just looking for ideas

itay livni · Answer

< Rob Fowler> Not sure if this is what you are after but prefect allows you to limits

Jeremiah · Answer

< Rob Fowler> indeed Prefect Cloud s enterprise tiers include flexible task concurrency limits that can be enforced globally across all clusters flows and are based on task `tags`

Jeremiah · Answer

We will be bringing those to lower tiers of Cloud in the near future

Rob Fowler · Answer

this is for a POC we don t have prefect in production yet I am sure if it works out we have other use cases I ll be looking at some license just not cloud as we are a cloud provider ourselves

Rob Fowler · Answer

just back from dropping the kids off So I see I can tag the task then set a task limit with ```update task tag limit```

Rob Fowler · Answer

awesome Out of the box functionality

Jeremiah · Answer

Yes as a matter of fact I believe you ll want to call `update task concurrency limit` the other one will work but is a deprecated route

Rob Fowler · Answer

in the docs the tag one is described no notes on it being depreciated yet

Rob Fowler · Answer

finally got back to this seems it s cloud only back to the dask idea This won t work in cloud as it needs to connect to winrm hosts

Rob Fowler · Answer

```executor = LocalDaskExecutor pool=ThreadPool num workers=N ``` Not thanking you for saying in the doco that kwargs are Dask config arguments when actually you also mess with num workers independently So ```executor = LocalDaskExecutor pool=ThreadPool N ``` as expected does not work