Hi! I don't know if this happened to any of you. I...
# ask-communityj
jcozar
06/07/2023, 7:00 AMHi! I don't know if this happened to any of you. I am running a prefect agent in AWS ECS (fargate) as a service. Today the agent raised an error, and after that it is still alive but it does not work (work queue is healthy but agent does not run any flow). The error seems to be like a network error or something like that.
Copy code
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/h2/connection.py", line 224, in process_input
func, target_state = self._transitions[(self.state, input_)]
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
KeyError: (<ConnectionState.CLOSED: 3>, <ConnectionInputs.SEND_HEADERS: 0>)
During handling of the above exception, another exception occurred:
...
httpx.LocalProtocolError: Invalid input ConnectionInputs.SEND_HEADERS in state
ConnectionState.CLOSED
Backing off due to consecutive errors, using increased interval of 120.0s.
Resetting backoff due to successful run.✅ 1
jcozar
06/07/2023, 7:04 AMI configured an automation to detect if a flow run is in pending state for 15 minutes or more. That's how I detected this problem. Then I restarted the task in AWS ECS, and then I had to cancel flow runs and retry them, because the agent raised these errors:
Copy code
06:54:21.039 | ERROR | prefect.agent - Flow run 'XXXX' does not have an infrastructure pid attached. Cancellation cannot be guaranteed.j
Jake Kaplan
06/07/2023, 1:40 PMHey, thanks for the report! What version of prefect are you on?
There are couple of fixes in the past month or so that should have resolved this.
https://github.com/PrefectHQ/prefect/issues/7442
https://github.com/PrefectHQ/prefect/pull/9267
j
jcozar
06/08/2023, 6:42 PMThe prefect version that is running the agent is
2.10.9🙌 1
2 Views