I’m using Prefect Cloud with Kubernetes and have an issue with a zombie job pod stuck in a “Pending” state (due to trying to allocate more resources than possible due to a configuration error). I can force delete it (kubectl delete <pod-name> --force) but it comes back immediately (age of the pod resets to 0s so I think it was deleted). There are no Prefect flows or k8s deployments associated with the pod and the link to the flow run in its k8s description gives a 404. How do I get rid of it?
Ryan Sattler
12/23/2021, 5:32 AM
Update: I found some zombie Prefect jobs via
kubectl get jobs
and deleted those, which seems to have fixed the issue.