Shruti Hande

12/09/2022, 9:13 AM
I am trying to executed concurrent tasks of flow from multiple agents with same work queue, but only single agent is picking the tasks. Is there a way to run these tasks on multiple agent with same queue? Using prefect - 2.6.4 #prefect-community #prefect-contributors

Mason Menges

12/09/2022, 4:57 PM
Hey @Shruti Hande Can you elaborate on your use case here a bit more? Agents are just responsible for submitting flow runs to the execution environment you have defined in the deployment for your flows, Multiple agents polling from the same work-queue may pull in the same flow runs but only one of them will actually execute it so it may just be a matter of timing. We did implement agent-level concurrency limits recently in prefect 2.7 which you could leverage to ensure a particular agent is only managing some pre-defined number of flow runs at any given time but but there isn't necessarily a way to force flow runs to split evenly between 2 agents polling for work from the same work-queue. These docs might be helpful as well for explaining this in more detail: •

Shruti Hande

12/13/2022, 8:50 AM
So multiple agents cannot distribute or split the task /flow runs from the same work queue?