https://prefect.io logo
Join Slack
Powered by
This message was deleted.
# ask-marvin
s
This message was deleted.
m
Your message was too long, here's your message at the allowed limit: services postgres image postgres 15 environment POSTGRES_USER prefect POSTGRES_PASSWORD ${POSTGRES_PASSWORD -prefect} POSTGRES_DB prefect volumes - postgres_data /var/lib/postgresql/data healthcheck test [ CMD pg_isready -U prefect -d prefect ] interval 10s timeout 5s retries 5 start_period 15s redis image redis 7 volumes - redis_data /data healthcheck test [ CMD redis-cli ping ] interval 10s timeout 5s retries 5 prefect-server image prefecthq/prefect 3-latest depends_on postgres condition service_healthy redis condition service_healthy environment PREFECT_API_URL http //${PUBLIC_IP} 4200/api PREFECT_API_DATABASE_CONNECTION_URL <postgresql+asyncpg //prefect> ${POSTGRES_PASSWORD -prefect}@postgres 5432/prefect PREFECT_MESSAGING_BROKER prefect_redis messaging PREFECT_MESSAGING_CACHE prefect_redis messaging PREFECT_REDIS_MESSAGING_HOST redis PREFECT_REDIS_MESSAGING_PORT 6379 PREFECT_SERVER_API_AUTH_STRING ${PREFECT_AUTH_STRING} PREFECT_SERVER_API_CSRF_PROTECTION_ENABLED true PREFECT_SERVER_API_HOST 0 0 0 0 ports - 4200 4200 command prefect server start --host 0 0 0 0 --no-services healthcheck test [ CMD python -c import urllib request; urllib request urlopen('<http //localhost 4200/api/health>' timeout=2) ] interval 20s timeout 10s retries 5 start_period 30s prefect-services image prefecthq/prefect 3-latest depends_on prefect-server condition service_healthy environment PREFECT_API_URL http //${PUBLIC_IP} 4200/api PREFECT_API_DATABASE_CONNECTION_URL <postgresql+asyncpg //prefect> ${POSTGRES_PASSWORD -prefect}@postgres 5432/prefect PREFECT_MESSAGING_BROKER prefect_redis messaging PREFECT_MESSAGING_CACHE prefect_redis messaging PREFECT_REDIS_MESSAGING_HOST redis command prefect server services start prefect-worker image prefecthq/prefect 3-latest depends_on prefect-server condition service_healthy environment PREFECT_API_URL http //${PUBLIC_IP} 4200/api PREFECT_API_AUTH_STRING ${PREFECT_AUTH_STRING} command prefect worker start --pool default restart on-failure volumes postgres_data redis_data worker timeout File /usr/local/lib/python3 12/site-packages/httpx/_client py line 1730 in _send_single_request response = await transport handle_async_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/site-packages/httpx/_transports/default py line 393 in handle_async_request with map_httpcore_exceptions() ^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/contextlib py line 158 in exit self gen throw(value) File /usr/local/lib/python3 12/site-packages/httpx/_transports/default py line 118 in map_httpcore_exceptions raise mapped_exc(message) from exc httpx ConnectTimeout An exception occurred Traceback (most recent call last) File /usr/local/lib/python3 12/site-packages/httpx/_transports/default py line 101 in map_httpcore_exceptions d File /usr/local/lib/python3 12/site-packages/httpx/_transports/default py line 394 in handle_async_request resp = await self _pool handle_async_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/site-packages/httpcore/_async/connection_pool py line 256 in handle_async_request raise exc from None File /usr/local/lib/python3 12/site-packages/httpcore/_async/connection_pool py line 236 in handle_async_request response = await connection handle_async_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/site-packages/httpcore/_async/connection py line 101 in handle_async_request raise exc File /usr/local/lib/python3 12/site-packages/httpcore/_async/connection py line 78 in handle_async_request stream = await self _connect(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/site-packages/httpcore/_async/connection py line 124 in _connect stream = await self _network_backend connect_tcp(*kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/site-packages/httpcore/_backends/auto py line
31 in connect_tcp return await self _backend connect_tcp( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/site-packages/httpcore/_backends/anyio py line 113 in connect_tcp with map_exceptions(exc_map) ^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/contextlib py line 158 in exit self gen throw(value) File /usr/local/lib/python3 12/site-packages/httpcore/_exceptions py line 14 in map_exceptions raise to_exc(exc) from exc httpcore ConnectTimeout The above exception was the direct cause of the following exception Traceback (most recent call last) File /usr/local/lib/python3 12/site-packages/prefect/cli/_utilities py line 44 in wrapper return fn(*args *kwargs) ^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/site-packages/prefect/cli/_types py line 159 in sync_fn return asyncio run(async_fn(*args **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/asyncio/runners py line 195 in run return runner run(main) ^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/asyncio/runners py line 118 in run return self _loop run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/asyncio/base_events py line 691 in run_until_complete return future result() ^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/site-packages/prefect/cli/worker py line 115 in start is_paused = await _check_work_pool_paused(work_pool_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/site-packages/prefect/cli/worker py line 180 in _check_work_pool_paused work_pool = await client read_work_pool(work_pool_name=work_pool_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 12/site-packages/prefect/client/orchestration/_work_pools/client
✅ thought for 250.0 seconds
Thanks for the details — this looks like a Prefect 3 worker timing out while trying to read the work pool from the API. That usually means the worker cannot reach the API, or the API isn’t ready/responding in time. Top causes in Docker Compose and how to fix: 1) Wrong API URL inside the worker - In Compose, the worker must use the service name on the Docker network, not localhost or your PUBLIC_IP. - Set: 
PREFECT_API_URL=<http://prefect-server:4200/api>
- Using 
http://${PUBLIC_IP}:4200/api
often fails from inside containers. 2) Server not listening on all interfaces - In Docker, the API must bind to 0.0.0.0 or other containers can’t reach it. - Set: 
PREFECT_SERVER_API_HOST=0.0.0.0
- If you pass a CLI flag, use 
--host 0.0.0.0
(not spaced out). 3) Startup ordering/healthchecks - Keep 
depends_on: {prefect-server: {condition: service_healthy}}
for the worker so it starts after the API is actually healthy. 4) Bad/malformed env values in the snippet - These should be exact: - DB URL: 
PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://prefect:${POSTGRES_PASSWORD:-prefect}@postgres:5432/prefect>
- Messaging: 
PREFECT_MESSAGING_BROKER=prefect_redis.messaging
and 
PREFECT_MESSAGING_CACHE=prefect_redis.messaging
(note the dot, not a space) - Redis host: 
PREFECT_REDIS_MESSAGING_HOST=redis
- If you’re using auth, 
PREFECT_API_AUTH_STRING
in the worker must match 
PREFECT_SERVER_API_AUTH_STRING
on the server. 5) Increase API timeout if your server is slow - Default is 60s. You can bump it: - 
PREFECT_API_REQUEST_TIMEOUT=120
6) Make sure the work pool exists - If you’re starting the worker with 
--pool default
, ensure the pool is created and not paused:
Copy code
prefect work-pool create default --type process
Quick checks from inside the worker container: - Can it resolve and reach the API?
Copy code
docker compose exec prefect-worker sh -c "apk add --no-cache curl || true; curl -v <http://prefect-server:4200/api/health>"
- Can it resolve the DNS name?
Copy code
docker compose exec prefect-worker ping -c 1 prefect-server
Minimal, corrected Compose snippets for the key parts: - Server
Copy code
prefect-server:
  image: prefecthq/prefect:3-latest
  depends_on:
    postgres:
      condition: service_healthy
    redis:
      condition: service_healthy
  environment:
    PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:${POSTGRES_PASSWORD:-prefect}@postgres:5432/prefect>
    PREFECT_SERVER_API_HOST: 0.0.0.0
    PREFECT_SERVER_API_PORT: 4200
    PREFECT_MESSAGING_BROKER: prefect_redis.messaging
    PREFECT_MESSAGING_CACHE: prefect_redis.messaging
    PREFECT_REDIS_MESSAGING_HOST: redis
    PREFECT_REDIS_MESSAGING_PORT: "6379"
    PREFECT_API_REQUEST_TIMEOUT: "120"
    # Optional: PREFECT_SERVER_API_AUTH_STRING: ${PREFECT_AUTH_STRING}
  command: prefect server start --no-services
  ports:
    - "4200:4200"
  healthcheck:
    test: ["CMD", "python", "-c", "import urllib.request as u; u.urlopen('<http://localhost:4200/api/health>', timeout=5)"]
    interval: 20s
    timeout: 10s
    retries: 5
    start_period: 30s
- Services (if running separately)
Copy code
prefect-services:
  image: prefecthq/prefect:3-latest
  depends_on:
    prefect-server:
      condition: service_healthy
  environment:
    PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:${POSTGRES_PASSWORD:-prefect}@postgres:5432/prefect>
    PREFECT_MESSAGING_BROKER: prefect_redis.messaging
    PREFECT_MESSAGING_CACHE: prefect_redis.messaging
    PREFECT_REDIS_MESSAGING_HOST: redis
    PREFECT_REDIS_MESSAGING_PORT: "6379"
    PREFECT_API_REQUEST_TIMEOUT: "120"
  command: prefect server services start
- Worker ``` prefect-worker: image: prefecthq/prefect:3-latest depends_on: prefect-server: condition: service_healthy environment: PREFECT_API_URL: http://prefect-server:4200/api PREFECT_API_REQUEST_TIMEOUT: "120" # Optional if you enabled auth on server:
# PREFECT_API_AUTH_STRING: ${PREFECT_AUTH_STRING} command: prefect worker start --pool default restart: on-failure ``` If the worker still times out: - Share the exact value of 
PREFECT_API_URL
in the worker container (after env substitution) - Run the curl test above from inside the worker and paste the result - Confirm the pool name you’re using and whether it exists and is unpaused Happy to review your full docker-compose.yml if you paste it here, and we’ll pinpoint the minimal changes needed.
2 Views
Open in Slack
PreviousNext
Powered by