hi guys! is there any way to limit flow run (not t...
# prefect-community
hi guys! is there any way to limit flow run (not task run) concurrency for agents?
There are no built in mechanisms to limit concurrency at a flow-run level in the agent at this time. If you are using Prefect Cloud, the closest feature is configuring task tag concurrency (https://docs.prefect.io/cloud/concepts/task-concurrency-limiting.html) --which I realize is not at a flow-run level like you are asking for. You could create your own Agent class that inherits from the DockerAgent (or another agent class of choice) to get the desired effect. Specifically, to implement flow-run throttling at the agent (by limiting the result of
https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/agent.py#L313), however, this would not limit concurrency, it would only serve to throttle how often flow runs are started. To limit concurrency you would need to monitor the number of running flow runs to influence the result of
. Assuming you are using the DockerAgent, this would mean monitoring the number of child processes (which is roughly equal to the number of flow runs be run by this agent): https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/docker/agent.py#L87
that's perfect! thanks
hi! just a quick followup; I ended up implementing it as you suggested, subclassing the
class and redefining the
method; works great!