Mitch
11/20/2025, 1:41 AMMarvin
11/20/2025, 1:41 AMMarvin
11/20/2025, 1:47 AMprefect worker ls
prefect work-pool ls
2) If no worker is running, start one
- Temporarily (local test):
export PREFECT_API_KEY="pnu_..." # Cloud only
prefect worker start --pool "<your-pool>"
- Recommended for EKS: deploy the Kubernetes worker via Helm (stays online even when Karpenter scales nodes):
values.yaml:
worker:
apiConfig: cloud
cloudApiConfig:
accountId: "<your-account-id>"
workspaceId: "<your-workspace-id>"
apiKeySecret:
name: prefect-api-key
key: key
config:
workPool: "<your-work-pool-name>"
type: kubernetes
# optional: jobNamespace: "prefect" # if you want jobs in a separate ns
serviceAccount:
create: true
Install:
kubectl create namespace prefect
kubectl create secret generic prefect-api-key -n prefect --from-literal=key=pnu_xxx
helm repo add prefect <https://prefecthq.github.io/prefect-helm>
helm repo update
helm install prefect-worker prefect/prefect-worker -n prefect -f values.yaml
Then:
kubectl get pods -n prefect
kubectl logs -f deploy/prefect-worker -n prefect
3) Verify your deployment points to the same pool (and queue)
prefect deployment ls -v
- The deployment’s work_pool_name must match the pool your worker is polling.
- If you created a custom work queue, either:
- Start the worker without restricting queues (polls all queues in the pool), or
- Start it with the specific queue name.
4) Check queue/deployment status and limits
- UI: Work Pools → your queue shows READY, not Paused; Concurrency limit not 0.
- UI: Deployments → your deployment is active (not paused).
- If needed, resume in the UI and/or increase concurrency.
5) Confirm API connectivity from the worker
- In worker pod logs you should see it “Polling work pool …”.
- If not, check env:
- Cloud: API key present via the secret; account/workspace IDs correct.
- Self-hosted: PREFECT_API_URL points to your server and is reachable from the cluster.
6) Karpenter considerations
- Your worker must be “always on” for the queue to stay READY. If Karpenter is scaling the worker away, the queue will show NOT_READY and runs will go Late. Pin the worker to a small, persistent node class or add a PDB/min replicas so it isn’t evicted during scale down.
If you can, share a few details and I’ll pinpoint the fix:
- Are you on Prefect Cloud or self-hosted?
- The work pool name your deployment uses
- How you’re running the worker (Helm? local? container?)
- Output of:
prefect deployment ls -v
prefect worker ls
kubectl get pods -n prefect
kubectl logs -n prefect deploy/prefect-worker | tail -n 50
Helpful docs:
- Workers concept: Workers
- Kubernetes how-to: Run flows on Kubernetes
- Manage work pools/queues: Manage work pools
- Prefect Helm chart (worker): prefect-worker Helm chart