Great question. Short version: Prefect doesn’t autoscale workers for you; with Azure Container Apps (ACA) you’ll use KEDA-based autoscaling. With multiple work queues, you can either scale each queue independently or scale a single worker that polls multiple queues based on aggregate backlog. Here are the patterns that work well. Assumptions - Prefect 3.x service workers (started with

prefect worker start

) - ACA with KEDA scaling Options to autoscale 1) One container app per queue (most control) - Run one worker per queue:

prefect worker start -p <your-pool> -q queue-a --limit 2

Repeat for

queue-b

, etc., each as its own Container App. - Attach a scale rule per app that tracks that queue’s backlog. - Pros: clean isolation and per-queue scaling behavior. Cons: more apps to manage. 2) Single worker polling multiple queues (simpler footprint) - Run a worker that listens to multiple queues:

prefect worker start -p <your-pool> -q queue-a -q queue-b --limit 4

- Autoscale the single Container App based on the combined backlog across those queues. - Pros: fewer moving parts. Cons: less granular scaling; polling overhead grows with queue count. How to drive scaling (metrics) KEDA/ACA needs a metric. Prefect doesn’t expose “queue backlog” directly as a scaler, so use one of these: A) Prometheus exporter + KEDA Prometheus scaler - Stand up a tiny sidecar service that queries the Prefect API for scheduled runs per work queue and exposes Prometheus metrics. Then point a KEDA Prometheus scaler at it. Example exporter (FastAPI) that returns backlog metrics per queue:

import os
import httpx
from fastapi import FastAPI, Response

PREFECT_API_URL = os.getenv("PREFECT_API_URL")
PREFECT_API_KEY = os.getenv("PREFECT_API_KEY")
WORK_POOL = os.getenv("WORK_POOL")

app = FastAPI()

HEADERS = {"Authorization": f"Bearer {PREFECT_API_KEY}"} if PREFECT_API_KEY else {}

async def list_queues():
    async with httpx.AsyncClient(base_url=PREFECT_API_URL, headers=HEADERS, timeout=15) as c:
        r = await c.get(f"/work_pools/name/{WORK_POOL}/queues")
        r.raise_for_status()
        return r.json()

async def count_backlog(queue_id: str) -> int:
    # Count scheduled-ish runs for a queue. Adjust states to your needs.
    payload = {
        "flow_runs": {
            "operator": "and_",
            "work_queue": {"id": {"any_": [queue_id]}},
            "state": {"type": {"any_": ["SCHEDULED", "LATE", "AWAITING_CONCURRENCY_SLOT"]}},
        },
        "limit": 0
    }
    async with httpx.AsyncClient(base_url=PREFECT_API_URL, headers=HEADERS, timeout=30) as c:
        r = await c.post("/flow_runs/filter", json=payload)
        r.raise_for_status()
        data = r.json()
        # Some deployments return an array of runs; others a "data" payload. Adjust as needed.
        return len(data if isinstance(data, list) else data.get("flow_runs", []))

@app.get("/metrics")
async def metrics():
    queues = await list_queues()
    lines = []
    for q in queues:
        backlog = await count_backlog(q["id"])
        name = q["name"].replace('"', '\\"')
        lines.append(f'prefect_queue_backlog{{queue="{name}"}} {backlog}')
    return Response("\n".join(lines) + "\n", media_type="text/plain")

- Deploy this alongside your worker (same Container App or a separate one), then have a KEDA Prometheus scaler (via ACA custom scale rule) query: - Queue-specific:

sum(prefect_queue_backlog{queue="queue-a"})

- Aggregate:

sum(prefect_queue_backlog{queue=~"queue-a|queue-b"})

- Scale when the query exceeds a threshold (e.g., 1 backlog item per replica target). B) Push custom metrics to Azure Monitor + Azure Monitor scaler - Periodically compute backlog per queue (same logic as above) and push as a custom metric to Azure Monitor. - Use ACA “azure-monitor” scale rule to scale on that metric per queue or aggregate. C) Coarse autoscaling on CPU/memory

- If your flows are resource-bound and you’re okay with coarse scaling, scale on CPU/memory. Combine with: - Work queue concurrency limits:

prefect work-queue set-concurrency-limit "queue-a" 5

- Worker concurrency limit:

prefect worker start -p <pool> -q queue-a --limit 5

- This doesn’t directly reflect backlog but works for many setups. Important notes and best practices - A single worker can poll multiple queues; polling overhead increases with queue count. Keep it reasonable (e.g., 3–10). - For responsiveness, you can tune worker polling/heartbeat env vars: -

PREFECT_WORKER_QUERY_SECONDS

(default ~10s) -

PREFECT_WORKER_HEARTBEAT_SECONDS

-

PREFECT_WORKER_PREFETCH_SECONDS

See Settings reference. - Healthchecks: start workers with a health endpoint for ACA liveness/readiness checks: - Docs: Worker healthchecks. - Work queue and worker concurrency: - Work queue concurrency limits: Work queues - Worker

--limit

for in-process concurrency: Workers Prefect docs - Workers: Workers - Work pools/queues: Work pools and queues - Healthchecks: Worker healthchecks - REST API overview (for building the exporter): REST API reference A few clarifying questions to tailor a concrete example for you: - Are you on Prefect Cloud or self-hosted Prefect Server? (I’ll adjust the API auth/URL in the exporter.) - Do you prefer per-queue Container Apps (each with its own rule) or a single app that scales on aggregate backlog? - Do you already run Prometheus in your environment, or would you prefer the Azure Monitor route?