Emma Rizzi

04/19/2023, 12:37 PM
Hello, I'm trying to run my worker inside a kubernetes pod to avoid restarting it manually everytime it crashes (every few hours so far) I followed this tutorials for agents and adapted it for worker, it launches correctly but the pod does not restart when the worker is unhealthy. Source problem seems that the prefect worker process does not exit when the worker is unhealthy, so kubernetes does not see it's failing. I tried to configure the livenessprobe based on a basic grep (didn't find a way for this because the process still runs when worker is unhealthy), then based on a API request (Marvin suggestions seems to be confused between agents and workers?) but so far I can't make the API requests work (404 on every request i try), and at this point I'm thinking there must be something easier?


04/19/2023, 12:54 PM
Agreed - the agent needs a /heartbeat endpoint

Emma Rizzi

04/19/2023, 1:03 PM
According to marvin agents have a /healthz endpoint already ? or maybe its from Prefect 1, but that would be great on workers too I'll need this for production soon, so i'm trying to implements the health check by checking the logs now 🙂