<@ULVA73B9P> ImageNotFound(HTTPError('404 Client E...
# ask-marvin
f
@Marvin ImageNotFound(HTTPError('404 Client Error: Not Found for url: http+docker://localnpipe/v1.4 I use this to deploy : deploy( name=flow_name, work_pool_name="docker-worker-pool", image=DockerImage( name=flow_name, tag=environment, dockerfile="data_flow.Dockerfile", rm=True # Removes intermediate containers ), push=False, Im using prefect 3
m
thought for 405.6 seconds
Short version: your Docker worker can’t find the image it’s supposed to run. In Prefect 3.x, if you deploy with push=False, the worker must be using the exact same Docker daemon that built the image. If it isn’t, you’ll get ImageNotFound. And that http+docker://localnpipe/v1.4 bit smells like a mis-set Docker API version on Windows too. Two problems for the price of one. Lovely. What’s likely happening - Different daemon: You built the image locally during deploy (push=False), but your Docker work pool’s worker is running on a different machine or a different Docker daemon. It tries to pull "name:tag" from a registry, can’t find it, and fails with ImageNotFound. - Windows Docker API mismatch: The URL showing v1.4 strongly suggests DOCKER_API_VERSION is set incorrectly. The Docker SDK will try that version and the daemon responds 404. On Windows you’ll see http+docker://localnpipe when using the named pipe. How to fix it Pick one of these paths: 1) You want to run workers on another machine/daemon (most common): - Set push=True and use a fully qualified image name that points to a registry (e.g., ghcr.io/your-org/your-image:tag or your-docker-username/your-image:tag). - Ensure the worker can pull the image (docker login on the worker if private). - Example (Prefect 3.x): from prefect import flow from prefect.deployments import deploy from prefect.deployments.runner import RunnerDeployment from prefect.docker import DockerImage @flow def my_flow(): pass deployment = RunnerDeployment( name="my-flow/dev", entrypoint="path/to/flows.py:my_flow", work_pool_name="docker-worker-pool", ) deploy( deployment, image=DockerImage( name="your-docker-username/my-flow", # use a registry-qualified name tag="dev", dockerfile="data_flow.Dockerfile", rm=True # passed through as a docker build kwarg ), push=True, # important: push so the worker can pull ) 2) You really want push=False: - Run the Prefect worker against the exact same Docker daemon that did the build (same machine, same Docker runtime). - Or pre-load the image on the worker host: docker load or build there first. 3) Fix the Windows Docker API version issue (if applicable): - Check and unset DOCKER_API_VERSION if it’s set to something ancient like v1.4. - PowerShell: $Env:DOCKER_API_VERSION; Remove-Item Env:\DOCKER_API_VERSION - CMD: echo %DOCKER_API_VERSION% and set DOCKER_API_VERSION= - Make sure Docker Desktop is running, and you’re running the worker in an environment that can reach the Docker named pipe (avoid crossing WSL2/Windows boundaries unless configured). Sanity checks - Confirm your pool is a Docker work pool: - Create, if needed: prefect work-pool create "docker-worker-pool" --type docker - Start a worker: prefect worker start --pool docker-worker-pool - On the worker host, verify the image exists or can be pulled: - docker pull ghcr.io/your-org/your-image:tag (if push=True) - docker images | findstr your-image (Windows) to see if it’s local (if push=False) - Make sure the image name you pass is the exact one the worker attempts to pull/run (name + tag). Using just name=flow_name without a registry means Docker will look on Docker Hub’s library/ namespace if it needs to pull. A note on your snippet - Using rm=True is fine; DockerImage accepts extra docker build kwargs via **build_kwargs. - push=False is what’s biting you unless the worker is on the same Docker daemon. If you share: - Where the worker is running (same machine or remote? WSL2 vs Windows host?), - The exact image name the worker is trying to pull (from worker logs), - Whether DOCKER_API_VERSION is set, I can be more precise and marginally less gloomy. Docs for reference (Prefect 3.x): - Docker deployments: https://docs-3.prefect.io/v3/how-to-guides/deployment_infra/docker - Creating deployments: https://docs-3.prefect.io/v3/how-to-guides/deployments/create-deployments
CLI help we verified: - prefect deploy --help - prefect work-pool create --help - prefect worker start --help