Hristo Mavrodiev
09/27/2025, 9:48 AMprefect worker start --pool 'test'
. Can i have the kubernetes worker always onMarvin
09/27/2025, 9:48 AMMarvin
09/27/2025, 9:50 AMprefect worker start --pool 'test'
in a shell, that process lives only as long as the terminal/session does.
You’ve got a few good options to keep a Kubernetes worker “always on”:
Option A (recommended): run the worker inside your Kubernetes cluster
- Deploy the Prefect Kubernetes worker as a Kubernetes Deployment (via Helm). Kubernetes will keep it running and restart it if it dies, independent of your terminal.
- High-level steps:
1) Add/update the Helm repo
helm repo add prefect <https://prefecthq.github.io/prefect-helm>
helm repo update
2) Create a namespace (optional)
kubectl create namespace prefect
3) Provide your Prefect API credentials (e.g., for Prefect Cloud) as a secret
kubectl create secret generic prefect-api-key \
--namespace prefect \
--from-literal=key=<YOUR_PREFECT_API_KEY>
4) Install the worker chart and configure it (work pool name, API URL/Key, etc.) via values.yaml
helm install prefect-worker prefect/prefect-worker --namespace prefect -f values.yaml
- Docs:
- Deploy a worker using Helm
- Kubernetes how-to guide
- Manage work pools
Option B: run it as a background service on a VM
- If you prefer to run the worker off-cluster (e.g., a VM with kubeconfig that can reach your cluster), use a process supervisor so it survives logouts and restarts:
- systemd service (reliable, restarts on failure)
- or tmux/screen/nohup for simpler use
- Examples:
- Quick and simple:
nohup prefect worker start --pool test >prefect-worker.log 2>&1 &
disown
- More robust (systemd) — create a unit like:
[Unit]
Description=Prefect Kubernetes Worker
After=network.target
[Service]
Environment=PREFECT_API_URL=<your_api_url>
Environment=PREFECT_API_KEY=<your_api_key>
ExecStart=/usr/bin/env bash -lc 'prefect worker start --pool test'
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
Then:
sudo systemctl daemon-reload
sudo systemctl enable --now prefect-worker
A couple of checks/tips:
- Make sure your work pool exists and is type “kubernetes”:
prefect work-pool create "test" --type kubernetes --paused
prefect work-pool resume "test"
Docs: Manage work pools
- The worker process will connect to Prefect using PREFECT_API_URL
and (for Cloud) PREFECT_API_KEY
. Set those in your Helm values or service environment.
- Workers are stateless in Prefect 3.x; you can restart/replace them without impacting running flow runs.
Would you like help with a concrete Helm values.yaml for your environment, or do you prefer the systemd/tmux route?