Short answer: most “slow pickup” cases aren’t about the number of workers — they’re caused by concurrency limits or filters preventing more runs from being eligible at once. Quick checklist to pinpoint the bottleneck Run these and share the outputs if you can: 1) Check the work pool state and limits

prefect work-pool inspect "<your-pool>"
prefect work-pool ls --verbose

- Ensure the pool isn’t paused and that its concurrency limit is high enough for your target parallelism. - If needed, raise it:

prefect work-pool set-concurrency-limit "<your-pool>" 100

Docs: Work pool CLI 2) Check work queue limits/filters and paused state

prefect work-queue ls
# then for each queue ID you use:
prefect work-queue inspect "<queue-id>"

- Look for queue-level concurrency limits or if the queue is paused. - If you set queue filters (e.g., tags), make sure they actually match your deployments’ tags. - To raise/clear a queue limit:

prefect work-queue set-concurrency-limit "<queue-id>" 100
prefect work-queue clear-concurrency-limit "<queue-id>"

- Preview scheduled work:

prefect work-pool preview "<your-pool>" --hours 1

Docs: Work queue CLI 3) Check global/tag-based concurrency limits If your flows or tasks use tags tied to a limit, runs will queue until slots free up.

prefect concurrency-limit ls
# remove or raise as needed
prefect concurrency-limit inspect "<tag>"
prefect concurrency-limit create "<tag>" 100
prefect concurrency-limit delete "<tag>"

Docs: Global/tag-based concurrency limits 4) Verify the runs are actually eligible now - If runs are scheduled in the future, on backoff, or within a scheduling window, workers will not pick them up yet.

prefect flow-run ls --state scheduled

- Look at the next scheduled time for these runs. Docs: Retries and backoff 5) Look at worker logs for the reason they’re not submitting Common messages: - “No eligible work found” - “Work pool concurrency limit reached” - “Work queue paused” - “Run not eligible due to concurrency limit” These messages usually point to the exact limiter. 6) Sanity checks and troubleshooting helpers - Make sure your 10 workers are all pointed at the same pool/queues you expect:

prefect worker start --pool "<your-pool>" --work-queue "<queue-name>"

- You can “peek” at what a queue would hand out (and trigger a poll) with:

prefect work-queue read-runs "<queue-id>"

Docs: Worker health checks, Cloud rate limits Why this happens even with 10 workers - Pool or queue concurrency limits: If a pool/queue limit is 10, you’ll never have more than 10 active runs across your workers. - Tag-based/global concurrency: If your runs share a tag with a limit (e.g., 5), only 5 will run at once, the rest will wait. - Queue filters or paused queues: Workers won’t see runs that don’t pass filters or if queues are paused. - Scheduling/backoff: Runs may be in a future time window or waiting on retry backoff. - Infrastructure capacity: For process workers on a single host, the OS/CPU/memory can bottleneck startup. For Docker/K8s workers, cluster capacity or image pull time can throttle throughput. If you share: - Your work pool name and type - Output of: -

prefect work-pool inspect "<pool>"

-

prefect work-queue ls

and

prefect work-queue inspect "<queue-id>"

-

prefect concurrency-limit ls

- A snippet of worker logs around the delay …I can point to the exact limiter and suggest the quickest fix.

Here’s the mental model for Prefect 3.x: Work Pools contain Work Queues; Workers poll a Pool (optionally restricted to specific Queues); Deployments target a Pool (optionally a specific Queue). Concurrency can be set at both Pool and Queue layers. - Work Pool - Top-level bucket of work (typed: process, docker, kubernetes, etc.). - Has a Pool-wide concurrency limit and pause/resume state. - Contains one or more Work Queues. - CLI:

prefect work-pool create "my-pool" --type process
    prefect work-pool inspect "my-pool"
    prefect work-pool set-concurrency-limit "my-pool" 50
    prefect work-pool preview "my-pool" --hours 1

- Docs: Work pools - Work Queue - Lives inside a Work Pool. - Can have an optional concurrency limit and priority. - Can be paused/resumed; can optionally use filters (e.g., tags) to accept only certain runs. - CLI:

prefect work-queue create "high" --pool "my-pool" --priority 1
    prefect work-queue set-concurrency-limit "<queue-id>" 20
    prefect work-queue inspect "<queue-id>"
    prefect work-queue ls

- Docs: Work queues - Worker - A process that polls a Work Pool for eligible runs and submits them to infrastructure. - By default, it polls the Pool; you can restrict it to one or more specific Queues to shard workload. - Multiple Workers can poll the same Pool/Queue; the server hands out distinct runs. - CLI:

prefect worker start --pool "my-pool"                  # poll all queues in the pool
    prefect worker start --pool "my-pool" --work-queue "high" --work-queue "default"

- Health/observability: check worker logs for reasons runs aren’t picked up (e.g., “pool/queue concurrency limit reached”, “no eligible work”). - Docs: Worker health checks - Deployment targeting - A Deployment must target a Work Pool; it may optionally specify a Work Queue in that pool. - If no Queue is specified, the run goes to the Pool’s default routing (typically the default queue). - Example (Prefect 3.x pattern):

from prefect import flow

    @flow
    def my_flow():
        ...

    # Deploy from source and point to a pool (and optional queue)
    my_flow.from_source(
        source="<https://github.com/org/repo>",
        entrypoint="flows/my_flow.py:my_flow",
    ).deploy(
        name="prod",
        work_pool_name="my-pool",
        work_queue_name="high",  # optional
        cron="*/5 * * * *",
    )

- Concurrency and how it affects pickup - Pool concurrency limit caps the total simultaneous runs across all queues in that pool. - Queue concurrency limit further caps runs within that specific queue. - Tag/global concurrency limits are separate and can also throttle pickup.

prefect concurrency-limit ls
    prefect concurrency-limit inspect "<tag>"

- Docs: Global/tag-based concurrency limits Putting it together - Deployments enqueue runs into a specific Queue (or the default) within a Pool. - Workers poll the Pool and, if restricted, only the Queues you specified with

--work-queue

. - The server only hands runs to workers when they are eligible: not paused, within schedule/backoff windows, and all applicable limits (Pool, Queue, tag/global) have available slots. If your workers seem slow to pick up runs, common culprits: - Pool/Queue concurrency limits too low for your target parallelism. - Workers restricted to a Queue that your runs aren’t targeting. - Queue paused or filters not matching your deployments’ tags. - Tag/global concurrency limits throttling availability. If you want, share: -

prefect work-pool inspect "<pool>"

-

prefect work-queue ls

and

prefect work-queue inspect "<queue-id>"

-

prefect concurrency-limit ls

…and I’ll map exactly where your runs are getting bottlenecked.