Hello team 👋,
I have an agent deployed in ECS and it is only running one flow at a time and I wanted to run multiple concurrently. This single agent approach is increasing latency in our flow run times for flows submitted in short interval space between them. How can I solve this issue? Thanks.
k
Kevin Grismore
11/15/2023, 2:41 PM
Hi Andre, I'm not sure what might be causing this. Are your deployments using ECS infra blocks, or are flows running directly on the agent?
First, I'd check to see how the resource metrics are looking on your agent, it may be under-resourced depending on the volume of flow runs it's handling. I'd also take a look at the concurrency limit settings everywhere they exist to ensure nothing is misconfigured. You may also want to investigate upgrading your agent to a worker.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.