Can someone help me figure out what causes these `UnixHTTPConnectionPool` errors in Prefect Cloud? T...

Sean Leakehe

12/30/2021, 4:32 PM

Can someone help me figure out what causes these

UnixHTTPConnectionPool

errors in Prefect Cloud? These happen seemingly at random. I have this particular flow set to retry 3 times and it I got this on all 3 retries.

Kevin Kho

12/30/2021, 4:34 PM

Hey @Sean Leakehe, I have never seen this before. Is the Prefect Cloud or Server?

Sean Leakehe

12/30/2021, 4:34 PM

Hey Kevin, this is Prefect Cloud.

Kevin Kho

12/30/2021, 4:36 PM

You’re using docker agent?

Sean Leakehe

12/30/2021, 4:38 PM

Yep

Kevin Kho

12/30/2021, 4:39 PM

Are you using a volume mount? Are you on a Mac?

Kevin Kho

12/30/2021, 4:40 PM

Have you seen this thread?

👀 1

Kevin Kho

12/30/2021, 4:41 PM

or you can try bumping up the read timeout setting?

Sean Leakehe

12/30/2021, 4:46 PM

All of our flows run on an EC2 running CentOS.

Sean Leakehe

12/30/2021, 4:46 PM

Sorry, I’m a bit new to this, the read timeout is a docker setting ,correct?

Kevin Kho

12/30/2021, 4:48 PM

Yeah

✅ 1

Kevin Kho

12/30/2021, 4:49 PM

Like this

🙏 1

Kevin Kho

12/30/2021, 4:50 PM

Seems like it could be a load thing? Are you starting up a lot of containers when you see this error?

Sean Leakehe

12/30/2021, 4:54 PM

I think there could be as many as 4 containers that get spun up at the same time.

Kevin Kho

12/30/2021, 4:55 PM

I think it’s hard to guess the impact of 4 containers without knowing the resources of the machine, but the comments of the StackOverflow link I sent seem to suggest bumping up resources may help (in addition to the timeout settings)?

Sean Leakehe

12/30/2021, 4:57 PM

Roger that. I’ll start out with testing an increased timeout. Thanks for the help!

👍 1

Sean Leakehe

01/07/2022, 4:07 PM

Looping back - We were able to increase the timeout by setting

--docker-client-timeout INTEGER

when starting the docker agent.

Kevin Kho

01/07/2022, 4:10 PM

Ah nice! Thanks for the info

👍 1

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.