Hi Prefect,Is there a way to limit parallel flows in Prefect Server?For example I have two flows:
• flow_B Resource intensive, long-running flow that is run as a dependent flow for flow_A• flow_A :
◦ Flow that is scheduled to run every
◦ Calls flow_B
◦ Run with
to limit parallelism to max
runs at a time. For one scheduled run, the dependent flow flow_B is limited to run max
at a time.
Now, I noticed that for a single run of flow_A, flow_B is indeed limited to a parallelism of
runs at one time.However, when a previous flow_A (and dependent flow_Bs) are still running), and a new flow_A, with new dependent flow_Bs are scheduled than more than
flow_Bs can run at the same time.For example with parallelism y=2:
• At time 1 there will be 2 flow_Bs runing.
Run flow_A 1:|--dependent flow_B 1:running|--dependent flow_B 2:running|--dependent flow_B 3:waiting