Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hey all, does Prefect 2 have flow concurrency limits? I’ve read about work queues but the description on Work Queue Concurrency isn’t particularly clear:
&gt; Each work queue can optionally restrict concurrent runs of matching flows.
Does this mean that if I have `Flow_A` and `Flow_B` running, they can run concurrently for any concurrency limit value?

yep it’s int he docs: <https://docs.prefect.io/concepts/work-queues/#work-queue-concurrency>

&gt;  Work queue concurrency<https://docs.prefect.io/concepts/work-queues/#work-queue-concurrency|¶>
&gt; Each work queue can optionally restrict concurrent runs of matching flows.
&gt; For example, a work queue with a concurrency limit of 5 will only release new work if fewer than 5 matching runs are currently in a `Running` or `Pending` state. If 3 runs are `Running` or `Pending`, polling the queue for work will only result in 2 new runs, even if there are many more available, to ensure that the concurrency limit is not exceeded.
&gt; When using the `prefect work-queue` Prefect CLI command to configure a work queue, the following subcommands set concurrency limits:
&gt; • `set-concurrency-limit` sets a concurrency limit on a work queue.
&gt; • `clear-concurrency-limit` clears any concurrency limits from a work queue.
&gt; 

in the latest version 2.1 flow’s are bound to work queues 1:1

So, if I want to limit a single flow, I need to send it to it’s own work queue? Do several work queues pose a scaling issue or not really? (I’m not using Prefect Cloud)

so if you start a thousand flows and your concurrency is set to 10 your agent(s) would only run 10 at a time