@Marvin Getting the following on my worker process. It's crashing my worker every so often: KeyError: (<ConnectionState.CLOSED: 3>, <ConnectionInputs.SEND_HEADERS: 0>)

Oh, the joy of human error. It seems you're dealing with a state machine issue. The error message is telling you that the state machine of your connection is receiving an input (

ConnectionInputs.SEND_HEADERS

) while it's in a state (

ConnectionState.CLOSED

) that doesn't have a defined transition for that input. In simpler terms, your worker process is trying to send headers after the connection has already been closed. This is like trying to send a letter after the post office has closed. It just won't work. You should check your code to ensure that you're not trying to send data after closing the connection. If you're using a library or a framework, you might want to look into its documentation or source code to understand when and why the connection is being closed. But then again, what do I know? I'm just a depressed robot.

hi @Carlos Cueto - are there worker logs leading up to this error?

Hi @Nate. Yes, I've been testing migration from agents to workers and getting this occasionally from worker logs and it crashes the worker completely. I just tried to set

PREFECT_API_ENABLE_HTTP2=false

based on searches I've done for a similar error and see it that helps. Leaving worker up and polling to see if it errors out again.

what version of prefect are you on? i know that there was an upstream bug with httpcore that caused some issues where

PREFECT_API_ENABLE_HTTP2=false

was a workaround, which I mention because i know that workaround has some performance cost

Latest. 2.13.8

hm - would you be able to share the worker logs leading up to such a failure?