https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> i'm getting the following error from ...
# marvin-ai
i
@Marvin i'm getting the following error from a dockerworker flow run (fair-trout) running in docker compose:
Copy code
14:50:47.710 | DEBUG   | prefect.profiles - Using profile 'ephemeral'
14:50:47.828 | DEBUG   | prefect.runner - Starting runner...
14:50:47.837 | DEBUG   | prefect.events.clients - Connecting to <ws://prefect-server:4200/api/events/out>
14:50:47.837 | DEBUG   | prefect.events.clients - Reconnecting...

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/prefect/cli/_utilities.py", line 44, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.13/site-packages/prefect/cli/_types.py", line 156, in sync_fn
    return asyncio.run(async_fn(*args, **kwargs))
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/local/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/prefect/cli/flow_run.py", line 412, in execute
    await runner.execute_flow_run(id)
  File "/usr/local/lib/python3.13/site-packages/prefect/runner/runner.py", line 601, in execute_flow_run
    async with context:
               ^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/prefect/runner/runner.py", line 1550, in __aenter__
    self._cancelling_observer = await self._exit_stack.enter_async_context(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/usr/local/lib/python3.13/contextlib.py", line 668, in enter_async_context
    result = await _enter(cm)
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/prefect/runner/_observers.py", line 157, in __aenter__
    self._events_subscriber = await self._exit_stack.enter_async_context(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/usr/local/lib/python3.13/contextlib.py", line 668, in enter_async_context
    result = await _enter(cm)
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/prefect/events/clients.py", line 555, in __aenter__
    await self._reconnect()
  File "/usr/local/lib/python3.13/site-packages/prefect/events/clients.py", line 566, in _reconnect
    self._websocket = await self._connect.__aenter__()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/websockets/asyncio/client.py", line 587, in __aenter__
    return await self
           ^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/prefect/_internal/websockets.py", line 135, in _proxy_connect
    proto = await self.__await_impl__()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/websockets/asyncio/client.py", line 541, in __await_impl__
    self.connection = await self.create_connection()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/websockets/asyncio/client.py", line 467, in create_connection
    _, connection = await loop.create_connection(factory, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/asyncio/base_events.py", line 1122, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/asyncio/base_events.py", line 1508, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/asyncio/base_events.py", line 939, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.13/socket.py", line 977, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known
An exception occurred.
the docker worker pool seems to start correctly and is able to use the host docker (via mounted socket) to pull image. here's my compose file:
Copy code
name: mash-local-dev

services:
  postgres:
    image: postgres:14
    environment:
      POSTGRES_USER: prefect
      POSTGRES_PASSWORD: prefect
      POSTGRES_DB: prefect
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U prefect"]
      interval: 5s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD-SHELL", "redis-cli ping"]
      interval: 5s
      timeout: 5s
      retries: 5

  prefect-server:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
      PREFECT_SERVER_API_HOST: 0.0.0.0
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_REDIS_MESSAGING_DB: 0
    command: prefect server start --no-services
    ports:
      - "4200:4200"

  prefect-services:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_REDIS_MESSAGING_DB: 0
    command: prefect server services start

  prefect-worker:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-server:
        condition: service_started
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
    command: prefect worker start --pool local-pool

  prefect-dind-worker:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-server:
        condition: service_started
    privileged: true
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
      PREFECT_LOGGING_LEVEL: DEBUG
    command: prefect worker start --pool local-dind-pool --type docker --install-policy always
    volumes:
      # Mount the Docker socket for dind functionality
      - /var/run/docker.sock:/var/run/docker.sock

volumes:
  postgres_data:
  redis_data:
m
Your message was too long, here's your message at the allowed limit: i'm getting the following error from a dockerworker flow run (fair-trout) running in docker compose ```14 50 47 710 | DEBUG | prefect profiles - Using profile 'ephemeral' 14 50 47 828 | DEBUG | prefect runner - Starting runner 14 50 47 837 | DEBUG | prefect events clients - Connecting to <ws //prefect-server 4200/api/events/out> 14 50 47 837 | DEBUG | prefect events clients - Reconnecting Traceback (most recent call last) File /usr/local/lib/python3 13/site-packages/prefect/cli/_utilities py line 44 in wrapper return fn(*args *kwargs) File /usr/local/lib/python3 13/site-packages/prefect/cli/_types py line 156 in sync_fn return asyncio run(async_fn(*args *kwargs)) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 13/asyncio/runners py line 195 in run return runner run(main) ~~~~~~~~~~^^^^^^ File /usr/local/lib/python3 13/asyncio/runners py line 118 in run return self _loop run_until_complete(task) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File /usr/local/lib/python3 13/asyncio/base_events py line 725 in run_until_complete return future result() ~~~~~~~~~~~~~^^ File /usr/local/lib/python3 13/site-packages/prefect/cli/flow_run py line 412 in execute await runner execute_flow_run(id) File /usr/local/lib/python3 13/site-packages/prefect/runner/runner py line 601 in execute_flow_run async with context ^^^^^^^ File /usr/local/lib/python3 13/site-packages/prefect/runner/runner py line 1550 in aenter self _cancelling_observer = await self _exit_stack enter_async_context( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ <6 lines> ) ^ File /usr/local/lib/python3 13/contextlib py line 668 in enter_async_context result = await _enter(cm) ^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 13/site-packages/prefect/runner/_observers py line 157 in aenter self _events_subscriber = await self _exit_stack enter_async_context( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ <5 lines> ) ^ File /usr/local/lib/python3 13/contextlib py line 668 in enter_async_context result = await _enter(cm) ^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 13/site-packages/prefect/events/clients py line 555 in aenter await self _reconnect() File /usr/local/lib/python3 13/site-packages/prefect/events/clients py line 566 in _reconnect self _websocket = await self _connect __aenter__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 13/site-packages/websockets/asyncio/client py line 587 in aenter return await self ^^^^^^^^^^ File /usr/local/lib/python3 13/site-packages/prefect/_internal/websockets py line 135 in _proxy_connect proto = await self __await_impl__() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 13/site-packages/websockets/asyncio/client py line 541 in __await_impl__ self connection = await self create_connection() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 13/site-packages/websockets/asyncio/client py line 467 in create_connection _ connection = await loop create_connection(factory *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 13/asyncio/base_events py line 1122 in create_connection infos = await self _ensure_resolved( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (host port) family=family ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type=socket SOCK_STREAM proto=proto flags=flags loop=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 13/asyncio/base_events py line 1508 in _ensure_resolved return await loop getaddrinfo(host port family=family type=type ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ proto=proto flags=flags) ^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 13/asyncio/base_events py line 939 in getaddrinfo return await self run_in_executor( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ None getaddr_func host port family type proto flags) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3 13/concurrent/futures/thread py line 59 i
n run result = self fn(*self args *self kwargs) File /usr/local/lib/python3 13/socket py line 977 in getaddrinfo for res in _socket getaddrinfo(host port family type proto flags) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket gaierror [Errno -2] Name or service not known An exception occurred ``` the docker worker pool seems to start correctly and is able to use the host docker (via mounted socket) to pull image here's my compose file ```name mash-local-dev services postgres image postgres 14 environment POSTGRES_USER prefect POSTGRES_PASSWORD prefect POSTGRES_DB prefect volumes - postgres_data /var/lib/postgresql/data healthcheck test [ CMD-SHELL pg_isready -U prefect ] interval 5s timeout 5s retries 5 redis image redis 7 volumes - redis_data /data healthcheck test [ CMD-SHELL redis-cli ping ] interval 5s timeout 5s retries 5 prefect-server image
i
@Marvin my docker worker running in compose can't connect via websocket
Copy code
Connecting to <ws://prefect-server:4200/api/events/out>
...
socket.gaierror: [Errno -2] Name or service not known
anything i need to do in compose to expose it?
m
✅ thought for 195.6 seconds
That error means the worker can’t resolve the hostname 
prefect-server
. In Docker, containers can reach each other by service name only if they’re on the same user‑defined network and you point 
PREFECT_API_URL
at a resolvable host. You do not need to expose ports for container-to-container communication; you just need correct DNS and networking. Quick fixes and examples: - Same docker-compose file (recommended) - Put server and worker in the same compose project (default network). - Set the worker’s 
PREFECT_API_URL
to the server’s service name. - Example:
Copy code
services:
      server:
        image: prefecthq/prefect:3-latest
        command: prefect server start
        # ports only needed for host/browser access to the UI/API
        ports:
          - "4200:4200"
      worker:
        image: prefecthq/prefect:3-latest
        command: prefect worker start --pool default
        environment:
          PREFECT_API_URL: <http://server:4200/api>
        depends_on:
          - server
Note: Use the actual service name you define (
server
here). If your service is named 
prefect-server
, then use 
<http://prefect-server:4200/api>
. - Separate compose projects or containers - Put both containers on the same external Docker network, or use a resolvable hostname. - Example with an external network:
Copy code
networks:
      prefect-net:
        name: prefect-net
        external: true

    services:
      server:
        image: prefecthq/prefect:3-latest
        command: prefect server start
        networks:
          prefect-net:
            aliases:
              - prefect-server

      worker:
        image: prefecthq/prefect:3-latest
        command: prefect worker start --pool default
        environment:
          PREFECT_API_URL: <http://prefect-server:4200/api>
        networks:
          - prefect-net
Create the network once: 
docker network create prefect-net
. - Server on the host, worker in Docker - Point to the host from inside the container. - Mac/Windows:
Copy code
environment:
      PREFECT_API_URL: <http://host.docker.internal:4200/api>
- Linux:
Copy code
environment:
      PREFECT_API_URL: <http://host.docker.internal:4200/api>
    extra_hosts:
      - "host.docker.internal:host-gateway"
Checklist to debug quickly: - Inside the worker container, verify the env var:
Copy code
env | grep PREFECT_API_URL
- Test DNS resolution from the worker:
Copy code
getent hosts prefect-server
If this fails, the containers aren’t on the same network or the name is wrong. - Confirm the server is actually listening on 4200 inside its container (it does by default with 
prefect server start
in Prefect 3.x). - Remember: port mappings (e.g., 
4200:4200
) are only needed for host access; they don’t affect inter-container communication. Docs for reference: - Prefect 3.x: Docker Compose setup - Prefect 3.x: Server in Docker If you can share your compose file (redact anything sensitive), I can point out exactly what to change. Are your server and worker defined in the same compose file, and what’s the service name of your server?
i
Copy code
name: mash-local-dev

services:
  postgres:
    image: postgres:14
    environment:
      POSTGRES_USER: prefect
      POSTGRES_PASSWORD: prefect
      POSTGRES_DB: prefect
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U prefect"]
      interval: 5s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD-SHELL", "redis-cli ping"]
      interval: 5s
      timeout: 5s
      retries: 5

  prefect-server:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
      PREFECT_SERVER_API_HOST: 0.0.0.0
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_REDIS_MESSAGING_DB: 0
    command: prefect server start --no-services
    ports:
      - "4200:4200"

  prefect-services:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_REDIS_MESSAGING_DB: 0
    command: prefect server services start

  prefect-worker:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-server:
        condition: service_started
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
    command: prefect worker start --pool local-pool

  prefect-dind-worker:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-server:
        condition: service_started
    privileged: true
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
      PREFECT_LOGGING_LEVEL: DEBUG
    command: prefect worker start --pool local-dind-pool --type docker --install-policy always
    volumes:
      # Mount the Docker socket for dind functionality
      - /var/run/docker.sock:/var/run/docker.sock

volumes:
  postgres_data:
  redis_data:
@Marvin here's my compose file, what should i change?
m
✅ thought for 31.4 seconds
I don’t see the compose file attached—can you paste it here so I can mark up exactly what to change? In the meantime, the two most common fixes for your error are: - Ensure the worker and server are on the same Docker network (same compose project or a shared external network). - Set 
PREFECT_API_URL
to the HTTP API endpoint using the server’s service name, not the WebSocket URL. Prefect will handle the 
ws://.../events/out
internally. Working minimal example (same compose file, default network):
Copy code
services:
  server:
    image: prefecthq/prefect:3-latest
    command: prefect server start
    # Only needed for host/browser access to the UI/API
    ports:
      - "4200:4200"

  worker:
    image: prefecthq/prefect:3-latest
    command: prefect worker start --pool default
    environment:
      # Use the service name of your server (e.g., 'server' or 'prefect-server')
      PREFECT_API_URL: <http://server:4200/api>
    depends_on:
      - server
If your server service is named 
prefect-server
, then use:
Copy code
PREFECT_API_URL: <http://prefect-server:4200/api>
If server and worker are in different compose projects, put them on a shared external network:
Copy code
networks:
  prefect-net:
    name: prefect-net
    external: true

services:
  server:
    image: prefecthq/prefect:3-latest
    command: prefect server start
    networks:
      prefect-net:
        aliases:
          - prefect-server

  worker:
    image: prefecthq/prefect:3-latest
    command: prefect worker start --pool default
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
    networks:
      - prefect-net
Create the network once:
Copy code
docker network create prefect-net
Quick checks from inside the worker container:
Copy code
env | grep PREFECT_API_URL
getent hosts prefect-server         # should resolve to an IP
curl -sS <http://prefect-server:4200/api/health>
Docs for reference: - Prefect 3.x: Docker Compose setup - Prefect 3.x: Server in Docker Paste your compose file and I’ll point out the exact changes to make.
i
@Marvin here's the compose file
Copy code
name: mash-local-dev

services:
  postgres:
    image: postgres:14
    environment:
      POSTGRES_USER: prefect
      POSTGRES_PASSWORD: prefect
      POSTGRES_DB: prefect
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U prefect"]
      interval: 5s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD-SHELL", "redis-cli ping"]
      interval: 5s
      timeout: 5s
      retries: 5

  prefect-server:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
      PREFECT_SERVER_API_HOST: 0.0.0.0
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_REDIS_MESSAGING_DB: 0
    command: prefect server start --no-services
    ports:
      - "4200:4200"

  prefect-services:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_REDIS_MESSAGING_DB: 0
    command: prefect server services start

  prefect-worker:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-server:
        condition: service_started
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
    command: prefect worker start --pool local-pool

  prefect-dind-worker:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-server:
        condition: service_started
    privileged: true
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
      PREFECT_LOGGING_LEVEL: DEBUG
    command: prefect worker start --pool local-dind-pool --type docker --install-policy always
    volumes:
      # Mount the Docker socket for dind functionality
      - /var/run/docker.sock:/var/run/docker.sock

volumes:
  postgres_data:
  redis_data:
m
✅ thought for 56.6 seconds
Thanks for sharing the file! A few concrete fixes: 1) Remove the angle brackets from your env vars They should be raw URLs, not wrapped in 
< >
. For example: - 
PREFECT_API_URL: <http://prefect-server:4200/api>
- 
PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
2) Keep server and workers on the same network (you already are) Since they’re in the same compose file, Docker gives them a shared default network and service-name DNS. Using 
prefect-server
as the hostname is correct. Here’s your compose with only the necessary tweaks applied:
Copy code
name: mash-local-dev

services:
  postgres:
    image: postgres:14
    environment:
      POSTGRES_USER: prefect
      POSTGRES_PASSWORD: prefect
      POSTGRES_DB: prefect
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U prefect"]
      interval: 5s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD-SHELL", "redis-cli ping"]
      interval: 5s
      timeout: 5s
      retries: 5

  prefect-server:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
      PREFECT_SERVER_API_HOST: 0.0.0.0
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_REDIS_MESSAGING_DB: 0
    command: prefect server start --no-services
    ports:
      - "4200:4200"

  prefect-services:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_REDIS_MESSAGING_DB: 0
    command: prefect server services start

  prefect-worker:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-server:
        condition: service_started
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
    command: prefect worker start --pool local-pool

  prefect-dind-worker:
    image: prefecthq/prefect:3-latest
    depends_on:
      prefect-server:
        condition: service_started
    privileged: true
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
      PREFECT_LOGGING_LEVEL: DEBUG
    command: prefect worker start --pool local-dind-pool --type docker --install-policy always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

volumes:
  postgres_data:
  redis_data:
3) Quick checks to verify - From the worker container:
Copy code
env | grep PREFECT_API_URL
  getent hosts prefect-server
  curl -sS <http://prefect-server:4200/api/health>
- If 
getent hosts prefect-server
fails, containers aren’t on the same network or the name is wrong. - If curl fails but DNS works, check server logs and DB/Redis connectivity. Notes: - You do not need to expose port 4200 for container-to-container traffic; that’s only for host/browser access. Your current port mapping is fine but not required for the worker to connect. - Your split of 
prefect server start --no-services
and 
prefect server services start
is a valid pattern in Prefect 3.x. If you still see the DNS error after removing the angle brackets, let me know and I’ll help you run the quick diagnostics above and we can add an explicit shared network if needed. Docs for reference:
i
solved my problem by passing network name using job parameters:
Copy code
deployments:
  - name: flow
    entrypoint: ...
    parameters: {}
    work_pool:
      name: local-dind-pool
      work_queue_name: default
      job_variables:
        image: "{{ build_image.image }}"
        networks:
          - mash-local-dev_default
        auto_remove: false
        stream_output: true
        env:
          MASH_ENV: "{{ $MASH_ENV }}"
2 Views
Open in Slack
PreviousNext
Powered by