<@ULVA73B9P> give me a docker-compose.yaml which h...
# marvin-ai
p
@Marvin give me a docker-compose.yaml which have docker type’s workpool
m
thought for 273.8 seconds
Here’s a ready-to-use docker-compose.yml that includes a Prefect 3 self-hosted Server and a Docker-type work pool worker. It also creates the pool automatically and mounts the Docker socket so the worker can launch flow-run containers. Notes: - The official
prefecthq/prefect:3-latest
image does not include the docker integration by default, so the worker service installs
prefect[docker]
at startup. - After this stack is up, your flows should target the
docker-pool
work pool. - If your flow-run containers need to reach the API, set the work pool’s base job template to attach the same compose network (e.g.,
prefect_network
) or point
PREFECT_API_URL
to a reachable host address. ``` version: '3.8' services: postgres: image: postgres:14 environment: POSTGRES_USER: prefect POSTGRES_PASSWORD: prefect POSTGRES_DB: prefect volumes: - postgres_data:/var/lib/postgresql/data healthcheck: test: ["CMD-SHELL", "pg_isready -U prefect"] interval: 5s timeout: 5s retries: 10 networks: - prefect_network redis: image: redis:7 volumes: - redis_data:/data healthcheck: test: ["CMD", "redis-cli", "ping"] interval: 5s timeout: 5s retries: 10 networks: - prefect_network # Prefect API/UI prefect-server: image: prefecthq/prefect:3-latest depends_on: postgres: condition: service_healthy redis: condition: service_healthy environment: # Database PREFECT_API_DATABASE_CONNECTION_URL: postgresql+asyncpg://prefect:prefect@postgres:5432/prefect # API PREFECT_SERVER_API_HOST: 0.0.0.0 PREFECT_SERVER_API_PORT: 4200 # Messaging (recommended) PREFECT_MESSAGING_BROKER: prefect_redis.messaging PREFECT_MESSAGING_CACHE: prefect_redis.messaging PREFECT_REDIS_MESSAGING_HOST: redis PREFECT_REDIS_MESSAGING_PORT: 6379 PREFECT_REDIS_MESSAGING_DB: 0 command: prefect server start --no-services ports: - "4200:4200" healthcheck: test: ["CMD", "python", "-c", "import urllib.request as u; u.urlopen('http://localhost:4200/api/health', timeout=2)"] interval: 15s timeout: 5s retries: 10 start_period: 60s networks: - prefect_network # Background services (schedules, concurrency, notifications, etc.) prefect-services: image: prefecthq/prefect:3-latest depends_on: prefect-server: condition: service_healthy environment: PREFECT_API_DATABASE_CONNECTION_URL: postgresql+asyncpg://prefect:prefect@postgres:5432/prefect PREFECT_MESSAGING_BROKER: prefect_redis.messaging PREFECT_MESSAGING_CACHE: prefect_redis.messaging PREFECT_REDIS_MESSAGING_HOST: redis PREFECT_REDIS_MESSAGING_PORT: 6379 PREFECT_REDIS_MESSAGING_DB: 0 command: prefect server services start networks: - prefect_network # One-shot init to create the Docker work pool if it doesn't exist prefect-init: image: prefecthq/prefect:3-latest depends_on: prefect-server: condition: service_healthy environment: PREFECT_API_URL: http://prefect-server:4200/api command: > sh -lc " uv pip install 'prefect[docker]' && (prefect work-pool create --type docker docker-pool || true) && prefect work-pool ls " restart: "no" networks: - prefect_network # Docker worker that pulls work from the docker-pool prefect-docker-worker: image: prefecthq/prefect:3-latest depends_on: prefect-init: condition: service_completed_successfully environment: # Point the worker at your API PREFECT_API_URL: http://prefect-server:4200/api # Optional tuning PREFECT_WORKER_PREFETCH_SECONDS: "10" PREFECT_WORKER_QUERY_SECONDS: "15" PREFECT_LOGGING_LEVEL: INFO # Mount Docker socket so the worker can launch containers volumes:
- /var/run/docker.sock:/var/run/docker.sock command: > sh -lc " uv pip install 'prefect[docker]' && prefect worker start --pool docker-pool --name docker-worker-1 " restart: unless-stopped networks: - prefect_network volumes: postgres_data: redis_data: networks: prefect_network: driver: bridge ``` How to use: - Start:
docker compose up -d
- UI: open http://localhost:4200 - Your Docker work pool name:
docker-pool
- Point your deployments to
docker-pool
If you prefer Prefect Cloud instead of self-hosted: - Remove the postgres, redis, prefect-server, and prefect-services services. - In
prefect-docker-worker
, set: -
PREFECT_API_URL
to your Cloud workspace API URL - Add
PREFECT_API_KEY
with a service account key - Keep the
/var/run/docker.sock
mount and the
uv pip install 'prefect[docker]'
step. Helpful docs: - prefect-docker (Docker worker) docs - Workers and work pools overview If you want, share whether you’re targeting Prefect Cloud or self-hosted, and I can tailor this compose (e.g., set the work pool’s base job template to attach the correct Docker network so flow-run containers can reach the API).