<@ULVA73B9P> I found this error when try to deploy...
# ask-marvin
p
@Marvin I found this error when try to deploy a flow with docker pool: from prefect import deploy from prefect.docker import DockerImage
Copy code
docker.errors.APIError: 400 Client Error for <http+docker://localhost/v1.52/containers/0be05e44b07b4e863990b357e9ea002c3283f242391a19bed2e48c48941834cc/start>: Bad Request ("failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: exec: "prefect": executable file not found in $PATH")
16:37:10.211 | INFO    | prefect.flow_runs.worker - Reported flow run '818d84b3-775f-45f6-b857-55108b804eaf' as crashed: Flow run could not be submitted to infrastructure:
APIError(HTTPError('400 Client Error: Bad Request for url: <http+docker://localhost/v1.52/containers/0be05e44b07b4e863990b357e9ea002c3283f242391a19bed2e48c48941834cc/start'))>

from src.flows.collect_to_parquete import collect_to_parquet_flow
if __name__ == "__main__":
    deploy(
        collect_to_parquet_flow.to_deployment(name="collect_to_parquet_flow"),  # pyright: ignore[reportArgumentType]
        work_pool_name="docker-pool",
        image=DockerImage(
            name="wwwaste-data-pipeline",
            tag="v2",
            dockerfile="deployments/docker/Dockerfile",
        ),
        build=True,
        push=False,
    )
Dockerfile
Copy code
FROM python:3.14-slim-trixie

# The installer requires curl (and certificates) to download the release archive
RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
    ca-certificates \
    libgomp1 \
    build-essential\
    gcc libpq-dev\
    && rm -rf /var/lib/apt/lists/*
# Download the latest installer
ADD <https://astral.sh/uv/install.sh> /uv-installer.sh

# Run the installer then remove it
RUN sh /uv-installer.sh && rm /uv-installer.sh

# Ensure the installed binary is on the `PATH`
ENV PATH="/root/.local/bin/:$PATH"
ENV export PYTHONPATH=$PYTHONPATH:.

WORKDIR /app
COPY pyproject.toml uv.lock ./
RUN uv sync
RUN uv run prefect --version
COPY . /app
m
thought for 122.3 seconds
Thanks for the details — the error tells us exactly what’s happening: Docker is trying to start your flow container by running
prefect ...
, but there’s no
prefect
executable on the container’s PATH at runtime: exec: "prefect": executable file not found in $PATH Why this happens with your Dockerfile -
uv sync
installs your project (including Prefect, if listed) into a virtual environment under
/app/.venv
. - You verify Prefect during build with
uv run prefect --version
(which runs inside that venv), but at runtime the Docker worker runs
prefect
directly — not via
uv run
. - Because
/app/.venv/bin
is not on PATH, the
prefect
CLI isn’t found when the container starts. Two solid ways to fix it A) Keep using uv + project venv (recommended) - Make sure Prefect is in your pyproject dependencies. - Put the venv’s bin directory on PATH so
prefect
is discoverable at runtime. Example Dockerfile:
Copy code
FROM python:3.12-slim

# System deps
RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
    ca-certificates \
    libgomp1 \
    build-essential \
    gcc \
    libpq-dev \
  && rm -rf /var/lib/apt/lists/*

# Install uv
ADD <https://astral.sh/uv/install.sh> /uv-installer.sh
RUN sh /uv-installer.sh && rm /uv-installer.sh
ENV PATH="/root/.local/bin:$PATH"

WORKDIR /app

# Install project deps (including prefect) into /app/.venv
COPY pyproject.toml uv.lock ./
RUN uv sync --frozen

# Make the project venv active for all runtime processes
ENV VIRTUAL_ENV=/app/.venv
ENV PATH="${VIRTUAL_ENV}/bin:${PATH}"

# Optional: set PYTHONPATH to your app dir
ENV PYTHONPATH="/app:${PYTHONPATH}"

# Copy the rest of your code
COPY . /app

# (Optional) Quick smoke test
RUN which prefect && prefect --version
Key points: - Ensure
prefect
is listed in your
pyproject.toml
dependencies. - The
ENV PATH="${VIRTUAL_ENV}/bin:${PATH}"
line is what makes
prefect
available at runtime. B) Install Prefect into the system Python instead - If you prefer not to rely on a venv at runtime, install Prefect system-wide so
prefect
is on PATH without extra env changes:
Copy code
# after setting up Python
RUN pip install --no-cache-dir 'prefect>=3.0.0'
# or with uv:
# RUN uv pip install --system 'prefect>=3.0.0'
If you go this route, also install your project dependencies system-wide instead of to a venv, or the Prefect process won’t see your project packages. Small corrections to your Dockerfile - This line is incorrect and creates an env var literally named "export PYTHONPATH":
Copy code
ENV export PYTHONPATH=$PYTHONPATH:.
Use:
Copy code
ENV PYTHONPATH="/app:${PYTHONPATH}"
- Consider using a supported Python version (e.g., 3.11 or 3.12). 3.14 may not be fully supported by all dependencies yet. Optional quick workaround (without rebuilding) - If you know the venv lives at
/app/.venv
, you can add its
bin
directory to PATH via your Docker work pool’s environment: - In the Docker work pool settings (Base job template) or via deployment
job_variables
, set:
Copy code
"env": {
      "PATH": "/app/.venv/bin:${PATH}"
    }
This works only if your image already has
/app/.venv/bin/prefect
present. Other notes - You’re using
push=False
. Make sure the Docker worker is running on the same host that built the image so it can find
wwwaste-data-pipeline:v2
. If the worker is remote, set
push=True
and use a registry. Would you share: - Your
pyproject.toml
dependencies (to confirm
prefect
is included)? - Where your Docker worker is running (same machine vs remote)? With that, I can tailor the exact Dockerfile and deployment settings.
p
Yeah, it's work thank you