<@ULVA73B9P> i have a working prefect image for do...
# ask-marvin
o
@Marvin i have a working prefect image for docker usage. i have also have prefect worker in kubernetes. what do i need to change in the image to make it run on my cluster? is it the kubernetes plugin or something else?
m
thought for 322.7 seconds
Great question. Short answer: you typically don’t need to change your flow image at all to run on a Kubernetes worker. You do NOT need the Kubernetes plugin in the flow image — that’s only required on the worker. What the flow image needs (for Prefect 3.x): - Prefect core (prefect>=3) and your flow’s dependencies - If you use flow.from_source(...) with a Git repo: git must be installed in the image, and your repo credentials must be available at runtime (e.g., via env vars or mounted secrets) - If you pull code from cloud storage at runtime (S3/GCS/Azure), include the matching SDKs/libs your pull step uses What the flow image does NOT need: - No prefect-kubernetes package - No Kubernetes Python client - No kubectl Why: The Kubernetes worker (which has prefect-kubernetes and the Kubernetes client) is responsible for creating the Job/Pod. Your flow container just runs the flow. Any code “pull steps” (e.g., git clone) happen inside the flow container at runtime, so git/SDKs need to be there if you use from_source or other storage. Entrypoint/CMD guidance: - Don’t override the entrypoint/CMD. The Kubernetes worker injects the command to run your flow (prefect flow-run execute). The default Prefect base image already handles this. Image registry access: - Ensure your image is in a registry your cluster can pull from. If it’s private, configure imagePullSecrets on your Kubernetes work pool or deployment. Example Dockerfile (works for Docker and Kubernetes workers):
Copy code
FROM prefecthq/prefect:3-latest

# Optional: if you use from_source with Git
# (prefect base images usually include git, but add it if your base doesn’t)
# RUN apt-get update && apt-get install -y git && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install -r requirements.txt

# If you bake code into the image (alternative to from_source)
# COPY flows/ /opt/prefect/flows/

# Keep the default entrypoint; the worker injects the run command
Docs for reference: - How to run flows on Kubernetes: https://docs.prefect.io/v3/how-to-guides/deployment_infra/kubernetes - How to retrieve code at runtime (from_source, storage): https://docs.prefect.io/v3/how-to-guides/deployments/store-flow-code - Work pools and workers overview: https://docs.prefect.io/v3/concepts/work-pools Quick sanity check questions so I can tailor guidance: - Are you using flow.from_source(...) to pull code at runtime, or do you bake your flow code into the image? - Is your image in a private registry that your cluster needs credentials for? - Any private Git repos involved that need credentials in the pod?
o
@Marvin but the container cant be compiled from git, as the system is airgapped, so we are constrained with delivering a fully functional flow image that the worker on the cluster must pull from the local registry.
m
thought for 160.7 seconds
Got it — for an airgapped cluster you’ll want to ship a self-contained “flow image” and avoid any runtime pulls. Here’s what to change and what to avoid: What to put in the flow image - Include: prefect>=3, all your flow’s Python deps, and your flow code itself. - Do NOT include: prefect-kubernetes or Kubernetes clients — those are only needed on the worker. - Do NOT rely on runtime installs: ensure the work pool is not setting
EXTRA_PIP_PACKAGES
(the base image will try to install them at startup, which will fail offline). - Don’t use
flow.from_source(...)
or any pull steps; bake the code into the image so the pod can run fully offline. Where to place code and how entrypoints resolve - Working directory in the Prefect base image is
/opt/prefect
. - Use a file-path entrypoint that matches the on-image layout, e.g. if you copy to
/opt/prefect/flows/my_flow.py
, set entrypoint
flows/my_flow.py:my_flow
. - Prefect adds the working directory and the script’s parent to
sys.path
, so relative imports under
/opt/prefect
will work. Example offline-friendly Dockerfile - If you mirror images internally, replace the base with your mirrored tag.
Copy code
FROM registry.local/prefecthq/prefect:3-latest
WORKDIR /opt/prefect

# Optional: install from a local wheelhouse
# COPY wheels/ /wheels
# COPY requirements.txt .
# RUN pip install --no-index --find-links=/wheels -r requirements.txt

# Or just vendor your deps directly if you have them in-repo
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy your code so the entrypoint path matches this layout
COPY flows/ /opt/prefect/flows/

# Keep the default entrypoint; the worker injects the command
Deploying without building/pushing via Prefect - Since you’ll build and push the image to your local registry outside Prefect, tell Prefect not to build/push:
Copy code
from prefect import flow

@flow
def my_flow():
    ...

my_flow.deploy(
    name="my-k8s-deploy",
    work_pool_name="kubernetes",
    image="registry.local/my-team/my-flow:1.2.3",
    build=False,
    push=False,
    job_variables={
        # ensure the cluster can pull from your local registry
        "imagePullSecrets": [{"name": "regcred"}],
        # optional: serviceAccountName, resources, env, volumes, etc.
    },
)
Kubernetes worker requirements and config - The worker (not your flow image) needs the Kubernetes bits (
prefect-kubernetes
and the k8s client). Your flow image does not. - Ensure your Kubernetes work pool or job template sets
imagePullSecrets
to your local registry secret and any required
serviceAccountName
,
resources
,
nodeSelector
, etc. - Make sure your pod has
PREFECT_API_URL
and
PREFECT_API_KEY
(if using Cloud) via env/secret so it can reach your Prefect server inside the airgapped network. Common gotchas in airgapped setups - A non-empty
EXTRA_PIP_PACKAGES
in the work pool will cause startup failures offline — clear it and bake everything into the image. - Private registry pulls require
imagePullSecrets
on the work pool, job template, or deployment. - If you used
from_source
before, remove those calls so the deployment has no
pull_steps
. Refs - How to run flows on Kubernetes: https://docs.prefect.io/v3/how-to-guides/deployment_infra/kubernetes - How to retrieve/store flow code (baked image vs from_source): https://docs.prefect.io/v3/how-to-guides/deployments/store-flow-code - Work pools and job variables: https://docs.prefect.io/v3/concepts/work-pools If you share: - your intended on-image path to the flow file, and - the name of your imagePullSecret, I can sketch the exact entrypoint and job_variables you’ll want.