https://prefect.io
Join Slack
Running a very basic flow in Prefect Cloud with prefect agent deployed in EKS. In our logging infra,...
# ask-community
g
Running a very basic flow in Prefect Cloud with prefect agent deployed in EKS. In our logging infra, I’m getting 
ERROR prefect.infrastructure.process - Process 'turquoise-woodlouse' exited with status code: -9
No more messaging beyond that. In Prefect Cloud UI, the flow runs indefinitely and doesn’t show this log message (level set to DEBUG). Anyone familiar with exit code -9? Can’t find any docs.
✅ 1
cc: @Taylor Curran
t
Hey Garren, could you share what docs you are working off of?
g
I can’t find any docs on this
r
I believe exit -9 tells us a Python subprocess was killed by the operating system, usually due to running out of memory.
upvote 1
The Python subprocess docs explain a little more here: https://docs.python.org/3/library/subprocess.html#subprocess.CompletedProcess.returncode So -9 is saying the process was terminated by Posix signal 9, i.e. SIGKILL
I think that explains why you're seeing that logged via the agent but not seeing anything in the cloud UI - SIGKILL terminates the flow process before it can report anything is wrong
g
🤦 thank you @Ryan Peden - it does explain the disconnect The pod itself, however, doesn’t crash with OOM 🤔 - will look into it
r
It might be because it's happening to a subprocess - IIRC you will only get OOM for a pod if a container's init process gets terminated by the kernel's OOM killer. So in this case, perhaps the subprocess running the flow gets terminated by the Linux kernel, after which the agent (and the pod it is a part of) continue running
z
g
@Zanie 👌
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by