Kiran
09/24/2025, 5:54 AMMarvin
09/24/2025, 5:54 AMMarvin
09/24/2025, 6:00 AMprefect work-pool inspect "<your-pool-name>"
Look for:
- paused = False
- concurrency_limit (see if it’s set)
- queues configured for that pool
- If your deployment is set to a specific work_queue_name
, ensure workers start with --work-queue "<that-queue>"
or are polling it.
2) Check for global concurrency limits by tag
- If your deployment/flows use tags, a tag-based limit may be throttling runs in a Pending “AwaitingConcurrencySlot” state.
- CLI:
prefect concurrency-limit ls
If you see limits that match your tags, either increase them:
prefect concurrency-limit create <tag> <new_limit>
or inspect/reset if slots are stuck:
prefect concurrency-limit inspect <tag>
prefect concurrency-limit reset <tag>
- Docs: Global concurrency limits
3) Check the work pool’s concurrency limit
- If a limit is set on the pool, only that many runs will be allowed concurrently, regardless of worker count.
- Increase or clear it if needed:
prefect work-pool set-concurrency-limit "<your-pool-name>" 200
or
prefect work-pool clear-concurrency-limit "<your-pool-name>"
- Docs: Work pools
4) Look at a “Pending” run’s state details
- In the UI, click a Pending run and open “State details”. If you see:
- “AwaitingConcurrencySlot …” → tag/global concurrency limit
- “Awaiting work pool concurrency” → work pool concurrency limit
- No worker found for queue/pool → pool/queue mismatch
5) Infra capacity/quotas
- If you’re using Docker/Kubernetes/ECS, workers may be up but your cluster/cloud may only admit a subset of jobs at a time. That typically shows up as some runs quickly starting while others sit Pending until capacity frees up.
- Check your infra for resource quotas or pending jobs.
6) Optional: subflow vs independent runs
- If you’re calling prefect.deployments.run_deployment
, the default is as_subflow=True
. That’s fine, but you can set as_subflow=False
to decouple from the parent flow’s state tracking if you prefer. Signature for reference:
from prefect.deployments import run_deployment
run_deployment(
name="path/to/deployment",
parameters={...},
as_subflow=True, # set to False if you want
work_queue_name="...", # ensure this matches your workers if you set it
tags=[...]
)
- API ref: run_deployment
If you can share a bit more, I can pinpoint it quickly:
- The work pool name and whether it shows a concurrency limit
- Whether your sub-deployment sets a work_queue_name
- What the “State details” say for one of the Pending runs
- Whether you’re using tags on the deployment/flows
- Which infrastructure type your workers use (Process, Docker, Kubernetes, ECS, etc.)
Links for reference:
- Work pools: docs
- Global concurrency limits: docs