Short answer: yes. There isn’t a hard-coded “queue length” limit in Prefect 3.x. A “queue” is just a set of flow runs in your API database that workers poll from. 20k queued flow runs is a scale we regularly see in production. The practical limit is your API/database throughput and how quickly workers can dispatch work to your compute. Key considerations and best practices - Prefect Cloud vs OSS - Cloud: The control plane is fully managed and scales well for large backlogs; this is usually the easiest path for 20k+ queued runs. - OSS: Use Postgres (not SQLite) for production. Size it appropriately (CPU/RAM, connection pool), and keep DB retention/logging sensible to avoid bloat. - Concurrency and fairness - Cap parallelism at the work pool to match your compute budget:

prefect work-pool set-concurrency-limit <your-pool> 200

- Use tag-based or global concurrency limits to protect shared systems:

prefect concurrency-limit create "heavy" 50

- Docs: Global concurrency limits, Tag-based concurrency limits, Work pool CLI - Worker scaling and execution model - Run multiple workers across nodes and use a push-based work pool that provisions per-run infrastructure (Kubernetes, ECS, Docker) so the worker process isn’t your bottleneck. - Start workers targeting your pool/queue(s):

prefect worker start -p <your-pool> -q <optional-queue>

- Work pools concept: Work pools in Prefect 3.x - Backlog planning - If each run takes ~2 hours, the drain time is roughly (queued_runs / max_parallelism) * 2h. For example, 20,000 runs at 200-way parallelism ≈ 200 hours (~8.3 days). - If possible, submit runs in batches or schedule them across a time window to avoid sudden spikes in DB/API load and underlying cloud API rate limits. - Observability and retention - Keep log verbosity and result storage reasonable to avoid unnecessary DB and object-store load. - For OSS, configure DB retention policies and vacuum/analyze to keep performance steady. - Known caveat (ECS push at very high volumes) - If you plan to submit hundreds/thousands of runs rapidly to an ECS push work pool, watch for cloud API rate limits and queueing latency. There’s an open report describing slow transitions to PENDING in a high-volume ECS push setting: Issue #18877. What we don’t publish: a fixed numeric “max queue size.” In practice, 20k queued runs is fine as long as your control plane (Cloud or OSS Postgres) and workers are sized for it. A few quick questions to tailor guidance - Are you on Prefect Cloud or OSS? - Which infrastructure do you want workers to launch (Kubernetes, ECS, Docker, VMs)? - How fast will you enqueue the 20k runs, and what max parallelism do you want? - Any per-tenant or per-endpoint fairness requirements we should enforce with concurrency limits? If you share this, I can suggest exact pool types, concurrency settings, and a worker layout to match your 2-hour heavy workloads.