Hi! Is there a clean way to limit the number of co...
# prefect-server
c
Hi! Is there a clean way to limit the number of concurrently executing runs of a flow? For example, let's say I have a flow that runs every 15 minutes. Sometimes the flow run takes 20 minutes, but I don't want to trigger the next flow run until after the last one succeeds (or at the subsequent 15 min interval).
One idea I had was to have the first task of the flow query the GraphQL api to see if there are any
flow_runs
in-progress. If so, short-circuit the flow (using
case(is_flow_running, True)
) Hoping there is something a little more elegant/built-in
k
Hi @Casey Green, flow concurrency is something we offer in Prefect Cloud. Otherwise you should have to short-circuit with something like you are suggesting and
raise ENDRUN
to end the flow run and then wait for the next flow run
c
oh cool, good to know! In prefect cloud is it programmatically configurable in python or is that something you have to set in the UI?
k
Yes it can be controlled with the GraphQL API.
update_flow_concurrency_limit
and
delete_flow_concurrency_limit
Mutations
c
👍 excellent, thanks @Kevin Kho!
Do y'all have any docs around features that are only available in Cloud?
k
There is the What plan should I choose? section here. Other than that, not at the moment unfortunately. You’re very welcome to schedule a sales call and discuss though.
c
Great thanks! I think we'll be doing that here in the next month or two.
k
Might be relevant
c
Oh interesting! This solves a slightly different problem, but I've been actually wondering if this was possible as well. I think this makes the DockerAgent a more viable production deployment because it allows you to limit the number of flows executing on a single agent.