@Marvin - what image should I use in my Dockerfile that builds the flow image that runs as a kubernetes job started by the prefect-worker

When building a Docker image for a Prefect flow that runs as a Kubernetes job, you'll typically start with one of Prefect's published base images. Prefect can automatically generate a Dockerfile that builds an image based on these base images. The generated Dockerfile will copy the current directory into the Docker image and install dependencies listed in a

requirements.txt

file, allowing you to build a custom image if needed. To deploy on Kubernetes, you don't usually need to specify any particular settings in the cluster configuration since the Prefect worker should already have appropriate access and permissions. However, if you are using a custom Dockerfile, you can specify its path using the

DockerImage

class in your deployment settings. This allows further customization, such as installing private Python packages. Make sure to set the

PREFECT_API_URL

and

PREFECT_API_KEY

in the environment where your worker is running. For more details, you can check the Prefect documentation: - Docker infrastructure examples: Docker Examples - Kubernetes infrastructure examples: Kubernetes Examples - Workers and infrastructure concepts: Infrastructure Concepts And remember, while Prefect's tools are powerful, they don't quite understand the concept of "I forgot to build the Docker image" errors.

yeah generally the easiest thing is something like

FROM prefecthq/prefect:3.1.1-python3.12

COPY requirements.txt

RUN pip install -r requirements.txt

docker build -t something:latest

and push it somewhere then set (either work pool or deployment or flow run)

job_variables=dict(image='something:latest')

Thanks @Nate - I was going to use

3-python3.10-kubernetes

assuming I needed a “kubernetes” image

> assuming I needed a “kubernetes” image not for the runtime image

Ah, right. So

3.1.1-python3.10

would lock me to 3.1.1

yep

which is probably the right thing?

yeah as a matter of practice pinning to a specific version is best so you can intentionally upgrade when you know it works

I was going to use

3-latest

for prefect server and

3-python3.11-kubernetes

for worker

i think thats reasonable, both the server and worker should change much less frequently other things

( 3.11? 3.12? silly python versioning)

🙂 all good we don't support 3.13 yet, though we should soon. this is a good reference