https://prefect.io
Join Slack
Hi! I'm using prefect v1 and I'm trying to activate zombie killer to clean up tasks that are failed...
# ask-community
n
Hi! I'm using prefect v1 and I'm trying to activate zombie killer to clean up tasks that are failed but are stuck in forever running state. From what I understood from this link, all I need to do to activate it is to add
Copy code
env={"PREFECT__CLOUD__HEARTBEAT_MODE": "thread"}
to my run config. I added that but the tasks are still being stuck. Is there anything else I need to configure in order to activate zombie killer?
c
Where do the tasks run?
n
As kubernetes jobs
On GCP
c
This is a common issue - if a task fails or gets restarted for any reason in kubernetes, Kubernetes natively willr estart the pod, but Prefect is not aware of this restart so it continues to see a task as “running”
https://github.com/PrefectHQ/prefect/issues/7116 is for Prefect 2, but applicable as well for Prefect 1
in short, there is not a good answer for this behavior at this moment
n
I see, thanks!
m
Hi also having issues with this - using spot IO on top. How can we get this fixed asap. Affects production pipelines that we have. Huge Blocker
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by