Hi Prefect, Is there a way to limit parallel flows in Prefect Server? For example I have two flows: • flow_B Resource intensive, long-running flow that is run as a dependent flow for flow_A • flow_A : ◦ Flow that is scheduled to run every

t

minutes ◦ Calls flow_B

x

times using

StartFlowRun

◦ Run with

LocalDaskExecutor

to limit parallelism to max

y

runs at a time. For one scheduled run, the dependent flow flow_B is limited to run max

y

at a time. Now, I noticed that for a single run of flow_A, flow_B is indeed limited to a parallelism of

y

runs at one time. However, when a previous flow_A (and dependent *flow_B*s) are still running), and a new flow_A, with new dependent *flow_B*s are scheduled than more than

y

*flow_B*s can run at the same time. For example with parallelism `y=2`: • At time 1 there will be 2 *flow_B*s runing.

Run flow_A 1:
  |--dependent flow_B 1: running
  |--dependent flow_B 2: running
  |--dependent flow_B 3: waiting

• At time 2 there will be 4 *flow_B*s runing.

Running flow_A-1:
  |--dependent flow_B-1: finished
  |--dependent flow_B-2: running
  |--dependent flow_B-3: running

Run flow_A-2:
  |--dependent flow_B-4: running
  |--dependent flow_B-5: running
  |--dependent flow_B-6: waiting

Is it possible to limit the parallelism of flow_B to max

y

over all scheduled runs?

Hi @Peter Roelants, there is flow concurrency limiting, but only on cloud unfortunately. I don’t there’s a way around this in this specific scenario.

Thanks for the reply. Is there any plan to add this feature to Prefect server in the future?

Unfortunately, no. Check this

The newly suggested fix (to limit runs per agent) as mentioned at the end would probably solve my use case though: https://github.com/PrefectHQ/server/issues/213 Thanks for the reference 🙂

Yes feel free to chime in on that issue.