Great question. In Prefect 3.x, a Docker work pool will always start a new container per flow run. If you want event-triggered runs without creating a container per event, the common pattern is: Run a long‑lived Prefect worker in a single Docker container, and point your deployments to a Process work pool. The worker reuses the same container and executes each flow run as a subprocess. Here’s how to set that up: 1) Create a Process work pool

prefect work-pool create my-process-pool --type process

2) Run a long‑lived worker in one Docker container Example Dockerfile:

FROM <http://ghcr.io/prefecthq/prefect:3-latest|ghcr.io/prefecthq/prefect:3-latest>
ENV PREFECT_API_URL=<your_api_url>
ENV PREFECT_API_KEY=<your_api_key>
CMD ["prefect", "worker", "start", "--pool", "my-process-pool"]

Or docker-compose:

services:
  prefect-worker:
    image: <http://ghcr.io/prefecthq/prefect:3-latest|ghcr.io/prefecthq/prefect:3-latest>
    environment:
      PREFECT_API_URL: <https://api.prefect.cloud/api/accounts/<acct>/workspaces/<ws>>
      PREFECT_API_KEY: ${PREFECT_API_KEY}
    command: prefect worker start --pool my-process-pool
    restart: always

3) Deploy your flows with event triggers to that pool Using the CLI from your project:

prefect deploy

In your prefect.yaml, point deployments to the process pool and define triggers. Example:

deployments:
  - name: user-welcome
    entrypoint: flows/user_welcome.py:send_welcome
    work_pool:
      name: my-process-pool
    triggers:
      - type: event
        name: on-user-created
        match:
          prefect.resource.id: "webhook:user.created"
        parameters:
          user_id: "{{ event.payload.user_id }}"
  - name: order-processor
    entrypoint: flows/order_processor.py:process_order
    work_pool:
      name: my-process-pool
    triggers:
      - type: event
        name: on-order-created
        match:
          prefect.resource.id: "webhook:order.created"
        parameters:
          order_id: "{{ event.payload.order_id }}"

Notes and options - You can run many deployments and triggers against the same process pool; the single worker container will handle them without spawning more containers. - Set a safe concurrency for the pool to match your container’s capacity:

prefect work-pool set-concurrency-limit "my-process-pool" 10

- If you truly need per-run container isolation, keep a Docker work pool—but that will create a new container per run by design. Docs for deeper reference: - Work pools: Work pools - Workers: Workers - Event/deployment triggers: Creating deployment triggers and Event triggers - Customize job variables: Customize job variables A couple of clarifying questions so I can tailor this: - Are you on Prefect Cloud or self-hosted? - Roughly how many events per minute and desired concurrency? - Is running a single long‑lived worker container acceptable for your environment/security model?

Got it — you need to handle bursts of ~100 event triggers/minute, run them in parallel, and “pause” runs for days for approval without keeping containers alive. Recommended architecture - Use a Process work pool with one or more long‑lived Prefect workers running inside Docker containers. This reuses the same containers for many runs (no per-event containers). - Define event triggers on your deployments to kick off runs. - In your flow, call suspend_flow_run(wait_for_input=...) to tear down infra and resume later when approval is received. Why this works - Process work pool: runs each flow as a subprocess inside your worker container. - Scale by running multiple workers on the same pool and setting a pool concurrency limit. - suspend_flow_run frees the worker and container while the run waits for input, so you can pause for days. Setup 1) Create a Process work pool

prefect work-pool create my-process-pool --type process

2) Run N long-lived worker containers Dockerfile:

FROM <http://ghcr.io/prefecthq/prefect:3-latest|ghcr.io/prefecthq/prefect:3-latest>
ENV PREFECT_API_URL=<your_api_url>
ENV PREFECT_API_KEY=<your_api_key>
CMD ["prefect", "worker", "start", "--pool", "my-process-pool"]

docker-compose:

services:
  prefect-worker:
    image: <http://ghcr.io/prefecthq/prefect:3-latest|ghcr.io/prefecthq/prefect:3-latest>
    environment:
      PREFECT_API_URL: <https://api.prefect.cloud/api/accounts/<acct>/workspaces/<ws>>
      PREFECT_API_KEY: ${PREFECT_API_KEY}
      PREFECT_WORKER_QUERY_SECONDS: "5"   # lower latency for bursts
    command: prefect worker start --pool my-process-pool
    restart: always

Scale horizontally as needed:

docker compose up -d --scale prefect-worker=5

3) Set pool concurrency to match desired parallelism

prefect work-pool set-concurrency-limit "my-process-pool" 200

- You can attach many workers to the same pool; they’ll share that concurrency cap. 4) Deploy your flows with event triggers pointing to the process pool Example prefect.yaml snippet:

deployments:
  - name: order-processor
    entrypoint: flows/order_processor.py:process_order
    work_pool:
      name: my-process-pool
    triggers:
      - type: event
        name: on-order-created
        match:
          prefect.resource.id: "webhook:order.created"
        parameters:
          order_id: "{{ event.payload.order_id }}"

Suspend for approval (days) and resume Flow code example:

from prefect import flow, get_run_logger
from prefect.flow_runs import suspend_flow_run
from prefect.input import RunInput

class Approval(RunInput):
    approve: bool
    reason: str | None = None

@flow
def process_order(order_id: str):
    logger = get_run_logger()
    # ... business logic ...
    needs_approval = True

    if needs_approval:
        <http://logger.info|logger.info>("Suspending for approval on order %s", order_id)
        # timeout=None allows waiting indefinitely
        decision = suspend_flow_run(wait_for_input=Approval, timeout=None)
        if not decision.approve:
            <http://logger.info|logger.info>("Order rejected: %s", decision.reason)
            return

    # Continue after approval
    <http://logger.info|logger.info>("Order %s approved, continuing", order_id)
    # ...

Ways to resume with input - UI: Open the paused flow run in Prefect UI and click Resume; you’ll get a form for the Approval fields. - API (programmatic):

from prefect.input import send_input
# Provide the paused flow run ID and the approval payload
send_input(
    run_input={"approve": True, "reason": "OK to proceed"},
    flow_run_id="<FLOW_RUN_ID>",
)

- You can trigger send_input from your own webhook handler or service once an approval arrives. Tuning for 100 events/min - Increase worker count and pool concurrency. Example: 5 workers x 40 concurrency each (OS limits permitting) to comfortably absorb bursts. - Keep PREFECT_WORKER_QUERY_SECONDS low (e.g., 5) to reduce pick-up latency. - If each run is CPU-heavy, size worker containers accordingly and consider multiple smaller workers for better scheduling. Key docs

- Work pools: Work pools - Workers: Workers - Event triggers: Event triggers - Creating deployment triggers: Creating deployment triggers - Interactive flows (pause/suspend/input): Interactive flows A couple quick questions so I can size this well: - Typical runtime per flow run and CPU/memory profile? - Any hard cap on total concurrency in your environment? - Where will approvals originate (UI, internal service/webhook, external system)?

Short answer: there’s no hard-coded Prefect limit. A Docker work pool will launch one container per flow run, and you can run as many concurrently as your host (CPU, memory, network) can handle. Prefect controls “how many at once” with concurrency limits; the ceiling is your infra capacity. How to control it - Pool-wide limit (most common): set an upper bound across all Docker workers in the pool

prefect work-pool set-concurrency-limit "my-docker-pool" 200

- Scale out workers: run multiple Docker workers against the same pool to reach the pool limit. Each worker will launch containers for runs, up to the pool’s cap. What to expect for IO-bound flows - IO-bound workloads allow high parallelism. On a modern VM, tens to low-hundreds of lightweight containers in parallel is typical, but the right number depends on: - Memory per container (RSS), network throughput, open file/connection limits, and Docker daemon overhead. - Practical approach: - Start with a modest pool limit (e.g., 50–100), ramp up gradually, and watch CPU (especially docker daemon), memory, network connections, and error rates. - If you need more concurrency than a single host handles comfortably, add more worker hosts or move to an orchestrator (Kubernetes/ECS). Tip: If you want to avoid per-event containers entirely, use a Process work pool with long-lived worker containers (runs execute as subprocesses). For long approvals, call

suspend_flow_run(...)

so no container stays running while the flow waits. Docs - Work pools: Work pools - Workers: Workers

Here’s how suspend/resume behaves in Prefect 3 for both Process and Docker workers. The key idea: suspend_flow_run tears down the runtime so you can wait (hours/days) without holding resources, then resume provisions fresh runtime to continue. Process work pool (runs as subprocesses on a long-lived worker) - When you call suspend_flow_run(...): - The flow run exits its subprocess; the worker frees its concurrency slot. - No Python process keeps running and no container is held open. - The worker container itself stays up (it’s your long‑lived poller). - When you resume (via UI Resume button or programmatic send_input/resume_flow_run): - The flow run is re-scheduled. - Any available Process worker for that pool will pick it up and start a new subprocess. - Your code resumes right after suspend_flow_run(...) and receives the input value you requested. - Implication: Perfect for long approvals; you pay zero “runtime” cost while waiting. Docker work pool (spawns a new container per run) - When you call suspend_flow_run(...): - The run’s container exits; the Docker worker frees its concurrency slot and the container is cleaned up. - No container remains while you wait. - When you resume: - The run is re-scheduled. - A Docker worker launches a new container from the deployment’s image. - Execution continues after suspend_flow_run(...), returning the input value. - Implication: You still get “no resources while waiting,” but each run (and each resume) uses a fresh container instance. What “continue after suspend” means - The Prefect engine persists the run’s state and graph. On resume, a new process/container starts the flow and continues after the suspend_flow_run call, returning the provided input. Any in-memory Python state not persisted elsewhere will be gone. Persist what you need via: - flow/task results (enable persistence where required) - parameters, variables, blocks, artifacts, or your own storage Code sketch for long approval

from prefect import flow
from prefect.flow_runs import suspend_flow_run
from prefect.input import RunInput

class Approval(RunInput):
    approve: bool
    comment: str | None = None

@flow
def process_io_job(job_id: str):
    # do IO work...
    # wait indefinitely for approval without holding resources
    decision = suspend_flow_run(wait_for_input=Approval, timeout=None)
    if not decision.approve:
        return
    # continue after approval
    # more IO work...

How to resume with input - UI: Open the flow run, click Resume, fill the generated form. - Programmatic:

from prefect.input import send_input
send_input(
    run_input={"approve": True, "comment": "approved"},
    flow_run_id="<FLOW_RUN_ID>",
)

Concurrency behavior while suspended - While suspended, the run does not count against worker or pool concurrency. - On resume, it consumes concurrency again (subprocess for Process, container for Docker). When to choose which - Use Process pool if you want to reuse a single long‑lived container and avoid per-run containers entirely. Suspend still tears down the per-run subprocess. - Use Docker pool if you want per-run isolation. Suspend ensures you don’t keep a container alive while waiting. Docs - Interactive flows (pause/suspend/input): Interactive flows - Work pools: Work pools - Workers: Workers If you share which pool you’re leaning toward, I can suggest sizing (worker count, pool concurrency) for your IO-bound workload and 100 events/min burst.