Hey all, I have a couple of Prefect flows - both l...
# ask-community
d
Hey all, I have a couple of Prefect flows - both local and Vertex runs. All of them are using docker storage. Now the flows work without any problem - but sometimes I do get the following error:
Copy code
. 23 January 2022 11:00pm]: 500 Server Error for <http+docker://localhost/v1.41/images/create?tag=2022-01-19t11-58-16-139521-00-00&fromImage=uxxx-flow>: Internal Server Error ("Get "<https://xxx>": context deadline exceeded")
Seems like the docker registry is unavailable for some reason for a short period of time - exactly when Prefect is trying to pull a flow image and run it. I have 2 questions that I couldn’t find any answers to: 1. Is there any configuration option I can use in order to make Prefect retry the image pulling on failure? 2. Can I configure Prefect to pull images only upon change (new flow version) and saving them locally? The way it works right now is that each run invokes an image pull. Thanks!
a
#1 I haven’t tried that but I believe you can add a container-level restart behavior as follows (docs):
Copy code
DockerRun(...,host_config={"restart_policy" = {"Name": "on-failure", "MaximumRetryCount": 3}})
#2 This is easier to configure on Kubernetes agent (via
imagePullPolicy
) rather than on Docker agent. Afaik, on docker agent you can only generally disable pulling flow images on the agent label by setting
no_pull=True
Docs
👀 1