Great question! In Prefect 3.x, a work pool is a server-side object—you create it once, then point as many workers as you want at it. In Docker Compose you typically handle this by: 1) Creating the work pool at startup (via a one-shot “init” service) 2) Running one or more worker services that attach to that pool 3) Optionally scaling the worker service to N replicas Important: remove the angle brackets around URLs in your env—those were placeholders. Use plain strings. Here’s a minimal update to your compose file that: - creates a “process” work pool named

local-process

- starts workers that attach to it - lets you scale to 2+ workers easily

services:
  postgres:
    image: postgres:14
    environment:
      POSTGRES_USER: prefect
      POSTGRES_PASSWORD: prefect
      POSTGRES_DB: prefect
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U prefect"]
      interval: 5s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD-SHELL", "redis-cli ping"]
      interval: 5s
      timeout: 5s
      retries: 5

  prefect-server:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
      PREFECT_SERVER_API_HOST: 0.0.0.0
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_REDIS_MESSAGING_DB: 0
    command: prefect server start --no-services
    ports:
      - "4200:4200"

  prefect-services:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_REDIS_MESSAGING_DB: 0
    command: prefect server services start

  # One-shot init to create the work pool (idempotent)
  prefect-pool-init:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-server:
        condition: service_started
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
    command: >
      bash -lc "
      for i in {1..30}; do
        prefect work-pool create local-process --type process && break || sleep 2;
      done;
      true"
    restart: "no"

  # Worker service that attaches to the pool
  prefect-worker:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-pool-init:
        condition: service_started
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
    command: prefect worker start --pool local-process
    restart: unless-stopped

volumes:
  postgres_data:
  redis_data:

How to run 2+ workers - Easiest: scale the

prefect-worker

service with Compose - docker compose up -d - docker compose up -d --scale prefect-worker=3 - Or define multiple worker services explicitly (same config, different service names), e.g.

prefect-worker-1

,

prefect-worker-2

, etc. All can point to

--pool local-process

. Notes and tips - Pool type:

process

runs your flow as a subprocess inside the worker container. This is great for local/dev or when your deployments pull code from source (e.g., Git). If you want each flow run in its own container, create a Docker-type pool instead and run Docker workers. - Work queues: you don’t need to specify queues unless you want to target specific queues; by default the worker will poll the pool’s queues.

- Make sure your deployments target the pool you created (e.g.,

flow.from_source(...).deploy(work_pool_name="local-process", ...)

). Docs - Work pools: Work pools - Process workers: Process work pools and workers - CLI (work pools): prefect work-pool - CLI (workers): prefect worker If you share how you plan to run your deployments (local process vs container-per-run), I can tailor the pool type and worker config further.