<@ULVA73B9P> We managed to run a flow in ECS but w...
# marvin-ai
f
@Marvin We managed to run a flow in ECS but we found that when two flows are executed at the same time, the other is waiting until the one finishes. How do we run them concurrently? My concurrency limit in work pool is UNLIMITED
m
thought for 595.3 seconds
Got it — if your ECS flow runs are serializing, there are a few common culprits. Assuming you’re on Prefect 3.x, here’s how to get true parallelism: First, a couple of quick clarifying questions to pinpoint the cause: - What state does the second run sit in while it’s “waiting”? (e.g., AwaitingInfrastructure, AwaitingConcurrency, Pending) - How are you starting your worker (exact
prefect worker start ...
command)? - Did you set a concurrency limit on the deployment (Max concurrent runs)? Checklist to enable concurrent runs 1) Check the deployment’s own concurrency limit - A deployment-level limit of 1 will serialize runs regardless of pool settings. - Inspect it:
Copy code
prefect deployment ls
  prefect deployment inspect "<flow-name>/<deployment-name>"
Look for
concurrency_limit
(should be None/Unlimited). - If you deployed in code, ensure you didn’t pass
concurrency_limit=1
to
flow.deploy(...)
. Docs: Deployments 2) Check for tag-based global concurrency limits - If your flow or tasks have tags that are limited, runs will queue.
Copy code
prefect concurrency-limit ls
If you see a limit on a tag your deployment uses, either delete it or raise it:
Copy code
prefect concurrency-limit delete <tag>
Docs: Global concurrency limits 3) Confirm the worker isn’t limiting you to 1 - If your worker was started with a concurrency cap (e.g.,
--limit 1
), it will only start one flow at a time. Check the command you used to start it and remove/increase any limit. - You can also run multiple workers against the same pool to scale out:
Copy code
prefect worker start -p <your-ecs-pool> -n worker-1
  prefect worker start -p <your-ecs-pool> -n worker-2
Docs: Workers, Work pools 4) Verify the work pool is truly unlimited - You mentioned it’s unlimited — good! Just to double-check:
Copy code
prefect work-pool inspect "<your-ecs-pool>"
Ensure no concurrency limit is set. 5) Confirm the ECS side isn’t the bottleneck - The Prefect ECS worker submits a separate RunTask per flow run. If the second run is stuck in “AwaitingInfrastructure”, check worker logs for AWS errors (e.g., Fargate vCPU quota, subnets without IPs, etc.). If AWS capacity is the issue, you’ll see ECS errors in the worker logs and need to raise quotas or adjust networking. - If you’re using a custom job template that creates an ECS Service with
desiredCount: 1
, that will also serialize work; use RunTask (the default) instead. Docs: ECS worker guide If you can share: - The run state of the “waiting” flow - The output of
prefect deployment inspect "<flow>/<deployment>"
- The output of
prefect concurrency-limit ls
- The exact worker start command and any worker logs around the time the second run is queued …I can pinpoint the exact gate and give you the minimal change to enable parallel runs.