Thanks for sharing the compose details and the traceback. The timeout is happening when the worker tries to make its very first API call to read the work pool: -

_check_work_pool_paused -> client.read_work_pool(...)

- httpx times out contacting the API This almost always means the worker cannot reach your API URL from inside the container. Top likely causes and fixes: 1) The API URL is not reachable from the worker container - In Docker, containers should talk to each other via service names on the default network, not the host’s PUBLIC_IP. - Fix: set the worker’s

PREFECT_API_URL

to the server service name:

PREFECT_API_URL=<http://prefect-server:4200/api>

Using

http://${PUBLIC_IP}:4200/api

often fails from inside the container. 2) Make sure you actually created the work pool - If the pool doesn’t exist, the worker’s first call will fail. Create it once (e.g., a process pool):

prefect work-pool create default --type process

Then start your worker:

prefect worker start --pool default

3) If you set server auth, set client auth too - You have

PREFECT_SERVER_API_AUTH_STRING

on the server. The worker must set:

PREFECT_API_AUTH_STRING=<same user:pass as the server auth string>

This is correct in your snippet, just make sure the env var actually resolves (no empty value). 4) Verify the server is listening correctly and healthy - Ensure it’s bound to all interfaces:

prefect server start --host 0.0.0.0 --no-services

- From the worker container, test connectivity:

curl -sS <http://prefect-server:4200/api/health>

If this times out, it’s network/addressing, not Prefect. 5) Double-check your Postgres URL and env expansions - The correct format is:

PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://prefect:${POSTGRES_PASSWORD:-prefect}@postgres:5432/prefect>

Note the use of

:-

for a default value. An invalid URL can prevent the server from starting properly. 6) Redis messaging (optional, for multi-worker setups) - Your values look fine, but ensure the integration is available. If you see an ImportError for

prefect_redis

, install the integration in the server/services image:

pip install "prefect-redis>=0.3"

- Required envs if you use Redis messaging:

PREFECT_MESSAGING_BROKER=prefect_redis.messaging
  PREFECT_MESSAGING_CACHE=prefect_redis.messaging
  PREFECT_REDIS_MESSAGING_HOST=redis
  PREFECT_REDIS_MESSAGING_PORT=6379

A tightened compose excerpt (only the key bits) to avoid timeouts: ``` services: prefect-server: image: prefecthq/prefect:3-latest environment: PREFECT_SERVER_API_HOST: "0.0.0.0" PREFECT_API_URL: "http://0.0.0.0:4200/api" PREFECT_API_DATABASE_CONNECTION_URL: "postgresql+asyncpg://prefect:${POSTGRES_PASSWORD:-prefect}@postgres:5432/prefect" PREFECT_SERVER_API_AUTH_STRING: "${PREFECT_AUTH_STRING:-}" # optional PREFECT_MESSAGING_BROKER: "prefect_redis.messaging" PREFECT_MESSAGING_CACHE: "prefect_redis.messaging" PREFECT_REDIS_MESSAGING_HOST: "redis" PREFECT_REDIS_MESSAGING_PORT: "6379" command: > prefect server start --host 0.0.0.0 --no-services ports: - "4200:4200" prefect-services: image: prefecthq/prefect:3-latest depends_on: prefect-server: condition: service_healthy environment: PREFECT_API_URL: "http://prefect-server:4200/api" PREFECT_API_DATABASE_CONNECTION_URL: "postgresql+asyncpg://prefect:${POSTGRES_PASSWORD:-prefect}@postgres:5432/prefect" PREFECT_MESSAGING_BROKER: "prefect_redis.messaging" PREFECT_MESSAGING_CACHE: "prefect_redis.messaging" PREFECT_REDIS_MESSAGING_HOST: "redis" command: prefect server services start prefect-worker: image: prefecthq/prefect:3-latest depends_on: prefect-server: condition: service_healthy environment: PREFECT_API_URL: "http://prefect-server:4200/api"

PREFECT_API_AUTH_STRING: "${PREFECT_AUTH_STRING:-}" # must match server if set command: prefect worker start --pool default ``` Quick checklist to resolve your timeout now: - Change worker

PREFECT_API_URL

to

<http://prefect-server:4200/api>

- Verify server is listening on 0.0.0.0 - Create the

default

work pool:

prefect work-pool create default --type process

- From the worker container, run

curl <http://prefect-server:4200/api/health>

and confirm it returns OK If it still times out after those changes, please share: - The exact value you’re using for

PREFECT_API_URL

in the worker - Output of

docker compose logs -f prefect-server prefect-worker

- Result of

docker compose exec prefect-worker curl -v <http://prefect-server:4200/api/health>

Reference notes (verified against Prefect 3 source): - Workers/clients use

PREFECT_API_URL

and optional

PREFECT_API_AUTH_STRING

- Server auth is

PREFECT_SERVER_API_AUTH_STRING

(clients must use

PREFECT_API_AUTH_STRING

with same value) - Postgres URL:

<postgresql+asyncpg://user:pass@host:5432/db>

- Redis messaging:

PREFECT_MESSAGING_BROKER=PREFECT_REDIS.messaging

, host via

PREFECT_REDIS_MESSAGING_HOST

etc.

Great questions. Here’s how to make your worker more robust and how to add a Docker worker in Prefect 3.x. Robust worker checklist (applies to any worker type) - Use a stable API URL reachable from containers:

PREFECT_API_URL=<http://prefect-server:4200/api>

- Add a restart policy in Docker:

restart: unless-stopped

- Enable the worker healthcheck server and add a Docker healthcheck: - Start with

--with-healthcheck

- Health probe: GET http://localhost:8080/health - Tune polling/heartbeat and concurrency: -

PREFECT_WORKER_QUERY_SECONDS=10

-

PREFECT_WORKER_HEARTBEAT_SECONDS=30

-

PREFECT_WORKER_PREFETCH_SECONDS=10

- Start with a limit:

--limit 5

- Turn on helpful logging for troubleshooting:

PREFECT_LOGGING_LEVEL=DEBUG

(drop to INFO in prod) - If you use server auth, set both: - On server:

PREFECT_SERVER_API_AUTH_STRING=<user:pass>

- On workers:

PREFECT_API_AUTH_STRING=<user:pass>

Example: a more robust process worker in Compose

services:
  prefect-worker:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-server:
        condition: service_healthy
    restart: unless-stopped
    environment:
      PREFECT_API_URL: "<http://prefect-server:4200/api>"
      PREFECT_LOGGING_LEVEL: "INFO"
      PREFECT_WORKER_QUERY_SECONDS: "10"
      PREFECT_WORKER_HEARTBEAT_SECONDS: "30"
      PREFECT_WORKER_PREFETCH_SECONDS: "10"
      # If your server enforces auth:
      # PREFECT_API_AUTH_STRING: "${PREFECT_AUTH_STRING}"
    command: >
      prefect worker start
      --pool default
      --limit 5
      --with-healthcheck
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request as u; u.urlopen('<http://localhost:8080/health>', timeout=1)"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s

Add a Docker worker 1) Install the Docker integration for the worker - The worker process must have

prefect-docker

installed. - Easiest is to build a tiny image that extends Prefect’s base image:

FROM prefecthq/prefect:3-latest
  RUN pip install "prefect[docker]"

Then use this image for your Docker worker service. 2) Create a Docker work pool

prefect work-pool create my-docker-pool --type docker

3) Give the worker access to the Docker Engine - If your worker runs in Docker, mount the Docker socket:

volumes:
    - /var/run/docker.sock:/var/run/docker.sock

- This lets it launch sibling containers for flow runs. 4) Ensure flow-run containers can reach your API - Put server and worker on the same user-defined Docker network and reference the server by service name in

PREFECT_API_URL

(e.g., `http://prefect-server:4200/api`). - Add that same network to your Docker work pool’s base job template so flow-run containers join it. Minimal base job template for the Docker work pool - You can set this when creating or by overwriting later:

prefect work-pool create my-docker-pool --type docker --base-job-template ./docker-base.json
  # or overwrite later
  prefect work-pool create my-docker-pool --type docker --base-job-template ./docker-base.json --overwrite

- Example `docker-base.json`:

{
  "job_configuration": {
    "image": "{{ image }}",
    "env": "{{ env }}",
    "networks": [
      "prefect-net"
    ],
    "volumes": "{{ volumes }}",
    "auto_remove": true,
    "stream_output": true
  },
  "variables": {
    "type": "object",
    "properties": {
      "image": {
        "type": "string",
        "default": "prefecthq/prefect:3-latest"
      },
      "env": {
        "type": "object",
        "default": {}
      },
      "volumes": {
        "type": "array",
        "items": {
          "type": "string"
        },
        "default": []
      }
    }
  }
}

- Use a named Compose network (e.g.,

prefect-net

) and attach server, worker, and flow-run containers to it. Compose snippet for a Docker worker ``` services: prefect-docker-worker:

image: your-org/prefect-docker-worker:latest # built from Dockerfile with prefect[docker] depends_on: prefect-server: condition: service_healthy restart: unless-stopped environment: PREFECT_API_URL: "http://prefect-server:4200/api" # PREFECT_API_AUTH_STRING: "${PREFECT_AUTH_STRING}" # if using server auth PREFECT_LOGGING_LEVEL: "INFO" PREFECT_WORKER_QUERY_SECONDS: "10" PREFECT_WORKER_HEARTBEAT_SECONDS: "30" PREFECT_WORKER_PREFETCH_SECONDS: "10" volumes: - /var/run/docker.sock:/var/run/docker.sock command: > prefect worker start --pool my-docker-pool --limit 5 --with-healthcheck healthcheck: test: ["CMD", "python", "-c", "import urllib.request as u; u.urlopen('http://localhost:8080/health', timeout=1)"] interval: 30s timeout: 10s retries: 3 start_period: 60s networks: - prefect-net prefect-server: # ... networks: - prefect-net networks: prefect-net: driver: bridge

Overriding Docker job settings per deployment
- You can override the base job template per deployment with `job_variables`:

flow.deploy( name="etl", work_pool_name="my-docker-pool", job_variables={ "image": "your-registry/your-image:tag", "volumes": ["/data/dataro"], "env": {"EXTRA_VAR": "1"} } )

Optional: Redis-backed messaging for multi-worker robustness
- If you plan to run multiple workers, enabling Redis messaging makes worker coordination more responsive:
  - Server/services:

PREFECT_MESSAGING_BROKER=prefect_redis.messaging PREFECT_MESSAGING_CACHE=prefect_redis.messaging PREFECT_REDIS_MESSAGING_HOST=redis PREFECT_REDIS_MESSAGING_PORT=6379 ``` - Ensure the

prefect-redis

integration is installed in your server/services image. If you’d like, share your current compose file and I can propose an exact patch adding a robust Docker worker service (with docker.sock, healthcheck, network, and the right env).