<@ULVA73B9P> is it possible to set a concurrency l...
# ask-marvin
m
@Marvin is it possible to set a concurrency limit on a multi-deployment level, e.g. for all deployments that have a certain tag?
m
thought for 182.6 seconds
Short answer: you can cap concurrency across multiple deployments, but not directly “by deployment tag.” You have two practical options in Prefect 3.x: 1) Limit concurrent flow runs across a group of deployments via a Work Pool - Put those deployments in the same work pool and set a pool concurrency limit. This caps how many flow runs from that pool can run at once (regardless of which deployment they came from). - CLI:
Copy code
prefect work-pool set-concurrency-limit "my-shared-pool" 3
- Docs: Work pool CLI reference 2) Limit concurrent task runs across deployments via tag-based concurrency - Add a shared tag to the tasks you want to limit, then create a concurrency limit for that tag. This caps concurrent task runs (not flow runs) across all flows/deployments using that tag. - Code:
Copy code
from prefect import flow, task

@task(tags=["gpu"])
def heavy():
    ...

@flow
def my_flow():
    heavy()
- CLI:
Copy code
prefect concurrency-limit create gpu 2
- Docs: Tag-based concurrency limits Important notes - There is no built-in “flow-run concurrency by tag” feature; deployment tags do not control concurrency. - Use a work pool limit for flow-run level control; use tag-based limits for task-run level control. Quick question to make sure we steer you right: Are you trying to limit how many flow runs can execute at once across those deployments, or limit certain resource-heavy tasks across flows?
i think this issue is about this, you might want to +1 it
m
Thanks @Nate, this is indeed what I am looking for.