Hi, my job code was suddenly stopped with `prefect.exceptions.TerminationSignal` exception. The flow...

Kosta

11/07/2023, 12:18 PM

Hi, my job code was suddenly stopped with

prefect.exceptions.TerminationSignal

exception. The flow run page shows this message under the title:

Copy code

Failed due to a(n) KeyError in the Prefect flow run worker. The error originated from not being able to determine the exit code for a job and reporting it as -1.

Flow definition in my code has

retries=5

configuration, but no retries were issued. Manual retry did solve the issue. Sounds like Prefect internal error. Did anyone encounter it? Anyone has a solution?

Bianca Hoch

11/08/2023, 10:37 PM

Hey Kosta, thanks for raising. Hmm- was your worker healthy around the time that this happened? Is this the full stack trace of the error?

Bianca Hoch

11/08/2023, 10:38 PM

What sort of infrastructure are you using?

Kosta

11/09/2023, 11:41 AM

The only mention of KeyError is in a one-liner at the to of Prefect UI:

Kosta

11/09/2023, 11:41 AM

Kosta

11/09/2023, 11:41 AM

(It is in completed state because of a manual retry later)

Kosta

11/09/2023, 11:42 AM

This is a screenshot of additional logs from Prefect UI

Kosta

11/09/2023, 11:42 AM

Kosta

11/09/2023, 11:43 AM

@Bianca Hoch, what do you want to know in terms of infrastructure? We are using Prefect workers.

Kosta

11/09/2023, 11:44 AM

This happened on Nov 6 at 21:10 UTC

4 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.