https://prefect.io
Join Slack
Is anyone else seeing jobs fail to start this morning using the hybrid hosting model? Our jobs are n...
# ask-community
m
Is anyone else seeing jobs fail to start this morning using the hybrid hosting model? Our jobs are not getting picked up this morning but we have not seen any errors in the logs on our Agents running in our k8s cluster and did not make any changes for our k8s environment or our Agents
✅ 1
b
Hi Matt, we received your inquiry in the service console as well. Would you mind giving us an example of a flow ID that is not being picked up?
A flow run ID would be sufficient as well.
m
Hi Bianca. something seems to have cleared it up now I think but we were in a bad state for at least 12 hours. Here was one we saw not get picked up this morning 0d1e49b0-f2b8-42e9-88ef-c318bc7328b8
gratitude thank you 1
b
Hi Matt, it appears that the flow run was cancelled at 
2022-06-29T14:47:21.405634+00:00
, which interrupted/cancelled all running tasks 
2022-06-29T14:47:21.406375+00:00
, and then resulted in a loss of heartbeat which prompted the zombiekiller service to fail the remaining tasks runs 
2022-06-29T14:48:24.749226+00:00
.
Outside of that, I don't see anything else unusual. To my knowledge, we haven't experienced any recent outages on our end either. We'll take note of this to see if any other factors could have caused this. Please do reach out if it happens again.
🙏 1
m
Thank you for the triage and follow up. I will let you know if the issue reoccurs.
👍 1
b
Here's a reference that may be helpful to rule out a service outage: https://prefect.status.io/
👍 1
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by