Thomas Pedersen
04/19/2022, 7:36 AMAnna Geller
Thomas Pedersen
04/19/2022, 12:15 PMAnna Geller
Thomas Pedersen
04/19/2022, 1:02 PMAnna Geller
Thomas Pedersen
04/19/2022, 2:43 PMAnna Geller
Thomas Pedersen
04/25/2022, 5:42 AMAnna Geller
Wouldn't docker storage create one container per flow though?yes, it would to make your docker image smaller, there are many ways of approaching this. One is shown here. Which problem do you try to optimize for here specifically - saving costs on container registry or saving disk space in your execution layer? Our flow runs seem to fail when our local dockerhub is down - the best approach would be to set up some automation e.g. via flow level state handler to take some action based on that, perhaps to notify you about the failure so that you could start a new flow run once your registry is back up? Actually, your idea is quite good - you could configure that on the one hand via --no-pull and also via restart_policy on
host_config
on the Docker run:
• restart_policy (dict) –
• Restart the container when it exits. Configured as a dictionary with keys:
◦ Name
One of on-failure
, or always
.
◦ MaximumRetryCount
Number of times to restart the container on failure.Thomas Pedersen
04/28/2022, 6:34 AMAnna Geller
Our biggest issue is the number of Python dependencies that we're usingfor that, packaging those into a python package can help - one simple example