Hi, team, pls help I'm trying to implement automat...
# ask-community
i
Hi, team, pls help I'm trying to implement automation to crash zombie flow runs as per https://github.com/PrefectHQ/prefect/releases/tag/3.1.8 I managed to get heartbeat going - but only for flows deployed by
.serve()
. I need this automation to work on flow runs from deployments, deployed with
prefect deploy ... prefect.yaml
. No luck, I tried to go through framework source code, find out how the mechanism for heartbeats works. If I'm not mistaken, heartbeat events are all implemented and used in
class Runner
. https://prefect-python-sdk-docs.netlify.app/prefect/runner/runner/ states that Runner is used for managing flow runs, deployed with
.serve()
and
serve()
. Now I feel lost. Is there a way to make heartbeat work for `prefect deploy`ed flows? And most important question, what is
Runner
, after all?
c
Hi Ihor; heartbeats should work the same for deployed flows under two conditions: • the submitted flow is running prefect version 3.1.8 • the submitted flow has
PREFECT_RUNNER_HEARTBEAT_FREQUENCY
set to an integer value greater than or equal to 30 Can you confirm that both of those are true and you are still seeing an issue? And to answer your other question: the
Runner
is essentially a process nanny for the flow run; this allows prefect to better manage the flow run's environment and capture certain failure modes better
i
@Chris White Sorry for late reply - yes, both of those conditions are true for both experiments. First experiment (serve) was on selfhosted prefect server without any workers. The second experiment was conducted on selfhosted server and Docker container as worker - I deployed flow to be ran on worker. I could see heartbeat events in the first case but no such events in the second case. Is there something else I could miss? Please advice, could it be some misconfiguration on our end or it might be some bug?
Also, about Runner - is it used for both served and deployed or only for served as in docs?
c
it's used for both served and deployed flows and follows the same code path; how did you set
PREFECT_RUNNER_HEARTBEAT_FREQUENCY
on your docker worker? and you are sure the docker image had 3.1.8 installed within it?
i
PREFECT_RUNNER_HEARTBEAT_FREQUENCY is set as env variable, 3.1.9 is installed within it. I ran
Copy code
logger.info("Version: " + subprocess.run(["prefect --version"], check=True, text=True, shell=True, capture_output=True).stdout)
logger.info("Config view: " + subprocess.run(["prefect config view"], check=True, text=True, shell=True, capture_output=True).stdout)
at the start of the flow, and it prints that:
image.png
Meanwhile, there is no events, only the event on flow start:
c
that's really interesting, i'm not sure what could be happening there especially if you're receiving other events; i think this would best be sorted out on GitHub - would you mind opening an issue describing the setup / the symptoms?
i
I'm afraid I cannot do that - setup belongs to company and I have no access nor permission to share it
I was able to replicate this behavior locally and created related github issue: https://github.com/PrefectHQ/prefect/issues/16515
thank you 1