Hi Prefects One of my flows crashed with an `httpx WriteErro

Question

Hi Prefects One of my flows crashed with an `httpx WriteError` After running for about 1h and completing 371 tasks the flow entered a crashed state with the following trace

Tim-Oliver · Answer

```Crash detected Execution was interrupted by an unexpected exception Traceback most recent call last File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 asyncio selector events py line 916 in write n = self sock send data BrokenPipeError Errno 32 Broken pipe The above exception was the direct cause of the following exception Traceback most recent call last File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore exceptions py line 8 in map exceptions yield File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore backends asyncio py line 49 in write await self stream send item=buffer File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages anyio streams tls py line 202 in send await self call sslobject method self ssl object write item File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages anyio streams tls py line 168 in call sslobject method await self transport stream send self write bio read File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages anyio backends asyncio py line 1297 in send raise self protocol exception anyio BrokenResourceError During handling of the above exception another exception occurred Traceback most recent call last File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpx transports default py line 60 in map httpcore exceptions yield File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpx transports default py line 353 in handle async request resp = await self pool handle async request req File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore async connection pool py line 253 in handle async request raise exc File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore async connection pool py line 237 in handle async request response = await connection handle async request request File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore async http proxy py line 313 in handle async request return await self connection handle async request request File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore async http2 py line 146 in handle async request raise exc File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore async http2 py line 110 in handle async request await self send request body request=request stream id=stream id File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore async http2 py line 223 in send request body await self write outgoing data request File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore async http2 py line 372 in write outgoing data raise exc File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore async http2 py line 360 in write outgoing data await self network stream write data to send timeout File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore backends asyncio py line 49 in write await self stream send item=buffer File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 contextlib py line 137 in exit self gen throw typ value traceback File path to miniconda3 envs test gfriedri em alignment flows lib python3 9 site packages httpcore exceptions py line 12 in map exceptions raise to exc exc httpcore WriteError The above exception was the direct cause of the following exception httpx WriteError```

Tim-Oliver · Answer

Just checked the agent logs and this error seems to happen from time to time However this is the first time that it actually crashed a flow

Bianca Hoch · Answer

Hi Tim thanks for reporting this here What version of Prefect are you using by chance

Bianca Hoch · Answer

Also would mind describing what the tasks are doing Are the tasks processing large amounts of data

Bianca Hoch · Answer

As a first troubleshooting step you could try bumping the `PREFECT API REQUEST TIMEOUT` You can preview the default for that value by running the following command in the CLI `prefect config view show defaults | grep TIMEOUT` You can adjust the value by running this line `prefect config set PREFECT API REQUEST TIMEOUT= 30 ` Also we recommend upgrading your Prefect version to 2 5 0 or above to benefit from the additional client stability fixes lt <https github com PrefectHQ prefect pull 7090> gt which were made

Justin Trautmann · Answer

hello bianca hello tim oliver were you able to find a mitigation strategy for this i m facing similar issues right now and increasing the api request timeout didn t help what do you think about adding `status HTTP 408 REQUEST TIMEOUT` to the list of retry codes in the base client