I have a couple of Prefect flows - both local and Vertex runs.
All of them are using docker storage.
Now the flows work without any problem - but sometimes I do get the following error:
. 23 January 2022 11:00pm]: 500 Server Error for <http+docker://localhost/v1.41/images/create?tag=2022-01-19t11-58-16-139521-00-00&fromImage=uxxx-flow>: Internal Server Error ("Get "<https://xxx>": context deadline exceeded")
Seems like the docker registry is unavailable for some reason for a short period of time - exactly when Prefect is trying to pull a flow image and run it.
I have 2 questions that I couldn’t find any answers to:1. Is there any configuration option I can use in order to make Prefect retry the image pulling on failure?
2. Can I configure Prefect to pull images only upon change (new flow version) and saving them locally? The way it works right now is that each run invokes an image pull.
8 months ago
#1 I haven’t tried that but I believe you can add a container-level restart behavior as follows (docs):