Hey all does Prefect 2 have flow concurrency limits I ve rea

Question

Hey all does Prefect 2 have flow concurrency limits I ve read about work queues but the description on Work Queue Concurrency isn t particularly clear gt Each work queue can optionally restrict concurrent runs of matching flows Does this mean that if I have `Flow A` and `Flow B` running they can run concurrently for any concurrency limit value

Clint M · Answer

yep it s int he docs <https docs prefect io concepts work queues work queue concurrency>

Clint M · Answer

gt Work queue concurrency<https docs prefect io concepts work queues work queue concurrency| > gt Each work queue can optionally restrict concurrent runs of matching flows gt For example a work queue with a concurrency limit of 5 will only release new work if fewer than 5 matching runs are currently in a `Running` or `Pending` state If 3 runs are `Running` or `Pending` polling the queue for work will only result in 2 new runs even if there are many more available to ensure that the concurrency limit is not exceeded gt When using the `prefect work queue` Prefect CLI command to configure a work queue the following subcommands set concurrency limits gt `set concurrency limit` sets a concurrency limit on a work queue gt `clear concurrency limit` clears any concurrency limits from a work queue gt

Clint M · Answer

in the latest version 2 1 flow s are bound to work queues 1 1

José Duarte · Answer

So if I want to limit a single flow I need to send it to it s own work queue Do several work queues pose a scaling issue or not really I m not using Prefect Cloud

Clint M · Answer

so if you start a thousand flows and your concurrency is set to 10 your agent s would only run 10 at a time