Federico Ferlito
11/17/2025, 11:57 AMMarvin
11/17/2025, 11:57 AMMarvin
11/17/2025, 12:00 PM/data:/data will try to bind a path that doesn’t exist on that host.
- On macOS/Windows with Docker Desktop, you also can’t bind arbitrary root paths like /data; only shared host directories (e.g., under /Users on macOS) are allowed unless explicitly configured.
What to check/fix
1) Confirm which Docker daemon your worker is using
- On the machine running the Prefect worker, run:
docker info
docker version
echo $DOCKER_HOST
If you see “docker-desktop” or your DOCKER_HOST points to your local machine, that’s the wrong daemon for binding /data on the VM.
- In the Prefect UI, open a recent flow run -> Job details and check the Docker “Mounts” reported. It should show Type: bind and Source: /data for a correct bind mount.
2) Ensure the worker uses the VM’s Docker daemon
- Run the Prefect worker on the VM that has /data mounted and ensure it uses the VM’s local Docker engine (no DOCKER_HOST pointing elsewhere).
- If you must run the worker off-box, set DOCKER_HOST/DOCKER_TLS_VERIFY/DOCKER_CERT_PATH to point to the VM’s Docker daemon (securely), so the containers are created on the VM.
3) If you actually intend to use Docker Desktop
- Use a host path that Docker Desktop can share (e.g., /Users/you/data:/data on macOS) and enable file sharing in Docker Desktop settings.
- Docker bind-mount docs: Bind mounts
- Docker Desktop file sharing (macOS example): File sharing settings
4) Optional: use a named volume bound to /data (on Linux VM)
- On the VM, create a named volume that binds to /data:
docker volume create \
--driver local \
--opt type=none \
--opt device=/data \
--opt o=bind \
data_volume
- In your Docker work pool’s volumes, use:
data_volume:/data
This still requires the worker to use the VM’s Docker daemon, but it avoids Docker auto-creating empty host dirs.
5) Double-check how volumes are set in the pool
- Make sure you’ve set the “volumes” value in the pool’s job variables (or in the decorator), not just the JSON schema defaults. In the Prefect UI, go to Work Pools -> your pool -> Edit -> Job variables and confirm it shows:
[
"/var/run/docker.sock:/var/run/docker.sock",
"/data:/data"
]
Quick sanity test on the correct host
- On the VM with /data:
docker run --rm -v /data:/data alpine sh -c "mount | grep ' /data '; ls -la /data"
You should see a bind mount and the expected contents.
If you share:
- Where your Prefect worker process is running
- The value of DOCKER_HOST for that worker (if set)
- A snippet of the “Mounts” section from docker inspect of a Prefect-launched container
…I can pinpoint the exact misrouting and suggest the minimal change.