I have a single flow where I want to limit concurr...
# ask-community
m
I have a single flow where I want to limit concurrency to one flow run at a time for business logic reasons. I see that in Prefect Cloud I can limit concurrency at a Flow level using labels. Currently we use labels to tag flows by team (business-intelligence, data-engineering, machine learning, etc) and by environment (uat, prod) for routing to the appropriate agent. Is there any other way of limiting flow concurrency based on something other labels (e.g. flow name and project)? I can make it work, but I'm just not crazy about the idea of adding another label that would specify flow name.
n
Hi @Mark McDonald - at the flow level that's the only way we handle concurrency at the moment; another option you do have is if there are specific tasks in the flow that are the real reason for the concurrency limit, you can add tags to those tasks and not interfere with your flow-level labels
m
got it. Thanks for the confirmation, @nicholas
n
I think you make a good argument that your expectations for labels sort of conflict with how they're used in the concurrency model though; if you want to open a discussion ticket and describe how you might like this to work, I think we'd be open to exploring some different methods here
m
will do
👍 1
n
That's great @Mark McDonald! Thank you
m
@nicholas - I'm trying to implement my own currency through a couple of calls to the graphql api. basically at the start of my flow I'm just going to query the graphql api, see if there are any other active runs and stop the current run if there are. I would do this by making these api calls: 1. Fetch the most recent flow runs, not including the current flow run (I need to figure out how to filter the current flow run out)
Copy code
{
  flow_run(where: {flow_id: {_eq: flow_id}}, limit: 1, order_by: {created: asc}) {
    id
  }
}
2. I'd then check the flow_run_state for the most recent flow run and make sure that status is success, failed, or cancelled
Copy code
{
  flow_run_state(where: {flow_run_id: {_eq: flow_run_id}} limit: 1, order_by: {timestamp: desc}) {
    state, timestamp
  }
}
Does this make sense to you? Is there a better way of querying the graphql api to see if there are any active runs for flow, regardless of flow version?
n
Hi @Mark McDonald - try something like this:
Copy code
query {
  flow_run(
    where: 
     { 
       flow_id: { _eq: <<flow_id from context>> }, flow_run_id: { _neq: <<flow_run_id from context>> },
state: { _in: ["Running", "Submitted"] } } ) 
  { 
    id
    state
  }
}
Sorry if the formatting is bad there, on mobile currently
That'll get you back any runs for the given flow that are in running or submitted states except for the current flow run; the presence of any of those should tell you not to proceed
m
awesome, this makes sense - thanks. I'll try it now
this works well, thanks again @nicholas
n
Great! 😁