Fugo Takefusa
09/16/2025, 3:49 AMMarvin
09/16/2025, 3:49 AMMarvin
09/16/2025, 3:54 AMprefect-docker
collection use the Docker Engine API (via the Docker Python SDK). Podman can expose a Docker-compatible socket, and Prefect can talk to it as if it were Docker.
What to do
- Install the Docker extra so Prefect can use the Docker worker:
pip install "prefect[docker]"
- Expose Podman’s Docker-compatible API and point Prefect to it. On Linux rootless Podman:
systemctl --user enable --now podman.socket
export DOCKER_HOST=unix:///run/user/$(id -u)/podman/podman.sock
Then verify:
python - <<'PY'
import docker, os
print("DOCKER_HOST=", os.getenv("DOCKER_HOST"))
print(docker.from_env().version())
PY
- Create a Docker work pool and start a worker (this worker will use Podman through the Docker API socket you set above):
prefect work-pool create my-podman-pool --type docker
prefect worker start --pool my-podman-pool
Notes and caveats
- Compatibility: This works because Podman implements a Docker-compatible REST API. It’s not officially tested by Prefect, so there can be edge cases.
- Auto-remove containers: Avoid `auto_remove=True`—it can cause races where logs/inspection fail if the container disappears too fast. See Issue 8421.
- Image builds:
- If you use Prefect’s Docker image build steps (prefect_docker.deployments.steps.build_docker_image
), they rely on Docker API/CLI behavior. Many users just build with podman build
themselves and set the deployment’s image
to that tag, skipping Prefect’s build step.
- If you do use the Prefect build step and it fails against Podman’s API, fall back to building with podman build
+ podman push
.
- macOS/Windows: Use Podman Desktop/Podman Machine and enable the Docker API socket, then set DOCKER_HOST
to the socket Podman exposes (often a Unix socket on macOS or a TCP endpoint via WSL2 on Windows). This setup is less battle-tested than Linux.
Docs and references
- Docker worker and collection reference: prefect-docker API reference
- Overview of work pools/workers: Work pools and workers guide
If you can share your OS and whether you want Prefect to also build images (vs. just run them), I can tailor the exact DOCKER_HOST
and deployment config for your setup.