<@ULVA73B9P> can prefect work with podman?
# ask-marvin
f
@Marvin can prefect work with podman?
m
thought for 315.3 seconds
Short answer: yes—via Podman’s Docker-compatible API. Context: In Prefect 3.x, the Docker worker and the
prefect-docker
collection use the Docker Engine API (via the Docker Python SDK). Podman can expose a Docker-compatible socket, and Prefect can talk to it as if it were Docker. What to do - Install the Docker extra so Prefect can use the Docker worker:
Copy code
pip install "prefect[docker]"
- Expose Podman’s Docker-compatible API and point Prefect to it. On Linux rootless Podman:
Copy code
systemctl --user enable --now podman.socket
export DOCKER_HOST=unix:///run/user/$(id -u)/podman/podman.sock
Then verify:
Copy code
python - <<'PY'
import docker, os
print("DOCKER_HOST=", os.getenv("DOCKER_HOST"))
print(docker.from_env().version())
PY
- Create a Docker work pool and start a worker (this worker will use Podman through the Docker API socket you set above):
Copy code
prefect work-pool create my-podman-pool --type docker
prefect worker start --pool my-podman-pool
Notes and caveats - Compatibility: This works because Podman implements a Docker-compatible REST API. It’s not officially tested by Prefect, so there can be edge cases. - Auto-remove containers: Avoid `auto_remove=True`—it can cause races where logs/inspection fail if the container disappears too fast. See Issue 8421. - Image builds: - If you use Prefect’s Docker image build steps (
prefect_docker.deployments.steps.build_docker_image
), they rely on Docker API/CLI behavior. Many users just build with
podman build
themselves and set the deployment’s
image
to that tag, skipping Prefect’s build step. - If you do use the Prefect build step and it fails against Podman’s API, fall back to building with
podman build
+
podman push
. - macOS/Windows: Use Podman Desktop/Podman Machine and enable the Docker API socket, then set
DOCKER_HOST
to the socket Podman exposes (often a Unix socket on macOS or a TCP endpoint via WSL2 on Windows). This setup is less battle-tested than Linux. Docs and references - Docker worker and collection reference: prefect-docker API reference - Overview of work pools/workers: Work pools and workers guide If you can share your OS and whether you want Prefect to also build images (vs. just run them), I can tailor the exact
DOCKER_HOST
and deployment config for your setup.