https://prefect.io logo
Title
g

Gustavo Silva

03/15/2021, 3:53 PM
I would like to ask how do I get more information to find the issue with a failed flow. In my logs I juts got
Process PID 266 returned non-zero exit code
k

Kyle Moon-Wright

03/15/2021, 5:32 PM
Hey @Gustavo Silva, You can certainly get more information by configuring more verbose logging by doing either of the following: • start the agent with 
show_flow_logs=True
 if using 
flow.run_agent()
 or 
--show-flow-logs
 if starting from the CLI •
set PREFECT__LOGGING__LEVEL="DEBUG"
 in the Agent environment I feel like I typically see this error when there is a mismatch between Prefect versions - an Agent should have a greater or equal version of Prefect than the flow version. Not saying this is the issue here, but might be worth looking into.
g

Gustavo Silva

03/15/2021, 5:35 PM
@Kyle Moon-Wright Awesome I will do this right away.
m

Miguel

03/15/2021, 5:40 PM
Thanks @Kyle Moon-Wright The flow logs were just saying:
Starting task run...
Eventually, we increase the memory of the agent, restarted the flow and it was able to complete so we suspect the process was killed by the OOM killer Is there a way to log the exit code?
k

Kyle Moon-Wright

03/15/2021, 5:52 PM
Hey @Miguel, You can certainly configure the logger by adjusting the settings found in the Prefect config or even add supplementary loggers as needed, but these details will be specific to your system so it’s hard to say which is best suited for your needs.
m

Miguel

03/15/2021, 5:55 PM
This would have to be done at the level of the agent. https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/local/agent.py#L115 I guess we can override the heartbeat function and include that
k

Kyle Moon-Wright

03/15/2021, 6:32 PM
Yeah this is a tough one, the Agent will never reach into your infrastructure due to the Hybrid Model even with a Server deployment - so OOM can be tough to discern amongst the noise, infra logs will probably be the most helpful in these cases.