Kate
10/17/2025, 11:45 AMMarvin
10/17/2025, 11:45 AMMarvin
10/17/2025, 11:49 AMpip install -U prefect-kubernetes
2) Create a Kubernetes work pool and start a worker:
prefect work-pool create "my-k8s-pool" --type kubernetes
prefect worker start --pool "my-k8s-pool"
3) Deploy your flow to that pool (Prefect 3.x):
from prefect import flow
@flow
def my_flow():
...
my_flow.from_source("<https://github.com/org/repo>", entrypoint="path/to/flow.py:my_flow").deploy(
name="k8s-deployment",
work_pool_name="my-k8s-pool",
)
What the package provides
- Kubernetes worker and job config: run each flow run as a Kubernetes Job with a customizable base job template.
- Blocks/credentials: e.g., KubernetesClusterConfig, KubernetesCredentials to configure cluster access.
- Convenience tasks/flows for K8s resources: create/read/update Jobs, Pods, Deployments, Services, and Custom Objects from your Prefect flows.
Docs and references
- Prefect + Kubernetes integration: prefect-kubernetes docs
- How to run on Kubernetes: Kubernetes worker/how-to
- Workers concept: Workers overview
- Helm chart (optional for deploying workers/server on K8s): prefect-helm
If you can share how you want your flow runs to execute (per-run K8s Jobs vs. subprocess inside a single pod), I can recommend the exact setup and template options.Kate
10/17/2025, 12:08 PMMarvin
10/17/2025, 12:08 PMMarvin
10/17/2025, 12:08 PMprefect worker start for a Kubernetes work pool must have prefect-kubernetes (and the Kubernetes Python client). Otherwise it can’t submit Jobs to your cluster.
- Flow image (the container that runs your code): Not required unless your flow code imports/uses prefect_kubernetes.* helpers. If your flow doesn’t call those, it only needs prefect (plus your app deps).
When you wouldn’t need it
- If you run a Process worker inside a pod (no per-run K8s Jobs), you don’t need prefect-kubernetes. You’re just running subprocesses inside that single pod.
- Hosting Prefect Server on Kubernetes does not require it.
How to tell in your setup
- If you can create a work pool with --type kubernetes and see a worker submit Kubernetes Jobs for each flow run, then the worker environment already has prefect-kubernetes installed (either baked into the image or installed by your Helm/chart customization).
Related links
- Integration overview: prefect-kubernetes docs
- How to run on Kubernetes: Kubernetes worker/how-to
- Workers concept: Workers overview
If helpful, share how your worker is deployed (image/Helm values), and I can point out exactly where prefect-kubernetes is being pulled in and whether you need it in your flow image too.Kate
10/17/2025, 12:19 PMworker:
image:
prefectTag: 3.4.4-python3.11-kubernetesMarvin
10/17/2025, 12:19 PMMarvin
10/17/2025, 12:20 PM3.4.4-python3.11-kubernetes = Prefect 3.4.4 + Python 3.11 + prefect-kubernetes installed.
- That package is what gives the worker the ability to submit each flow run as a Kubernetes Job/Pod (the Kubernetes worker type), watch the Job/Pod, stream logs, and apply the work pool’s base job template.
So in your setup, prefect-kubernetes is already being used by the worker. Without it, the worker could not be --type kubernetes and would not be able to create Jobs; you’d be limited to a Process worker (subprocess runs inside one pod).
Where you need prefect-kubernetes vs. not:
- Worker container: Required for Kubernetes workers (your tag already includes it).
- Prefect Server: Not needed.
- Flow run image: Only needed if your flow code imports/uses prefect_kubernetes helpers (e.g., create/read Pods/Jobs/Deployments from the flow or use the Kubernetes credentials/cluster config blocks). If you don’t import those, your flow image just needs Prefect + your app deps; the worker still creates the Job.
If you ever change the worker image to a non-kubernetes variant or a custom image, ensure it has prefect-kubernetes (or pip install prefect-kubernetes) or the worker won’t be able to submit K8s Jobs.
Helpful refs:
- Integration overview: prefect-kubernetes docs
- How to run on Kubernetes (work pools, base job template, etc.): Kubernetes worker/how-to
- Helm chart: prefect-helm
If you want, share your flow image details. I can confirm whether you need to add prefect-kubernetes there based on whether you’re calling any prefect_kubernetes.* utilities from your flows.