Thomas Pedersen
04/19/2022, 7:36 AMAnna Geller
04/19/2022, 10:01 AMThomas Pedersen
04/19/2022, 12:15 PMAnna Geller
04/19/2022, 12:45 PMThomas Pedersen
04/19/2022, 1:02 PMAnna Geller
04/19/2022, 1:51 PMThomas Pedersen
04/19/2022, 2:43 PMAnna Geller
04/19/2022, 5:40 PMThomas Pedersen
04/25/2022, 5:42 AMAnna Geller
04/25/2022, 5:17 PMWouldn't docker storage create one container per flow though?yes, it would to make your docker image smaller, there are many ways of approaching this. One is shown here. Which problem do you try to optimize for here specifically - saving costs on container registry or saving disk space in your execution layer? Our flow runs seem to fail when our local dockerhub is down - the best approach would be to set up some automation e.g. via flow level state handler to take some action based on that, perhaps to notify you about the failure so that you could start a new flow run once your registry is back up? Actually, your idea is quite good - you could configure that on the one hand via --no-pull and also via restart_policy on
host_config
on the Docker run:
• restart_policy (dict) –
• Restart the container when it exits. Configured as a dictionary with keys:
◦ Name
One of on-failure
, or always
.
◦ MaximumRetryCount
Number of times to restart the container on failure.Thomas Pedersen
04/28/2022, 6:34 AMAnna Geller
04/28/2022, 2:00 PMOur biggest issue is the number of Python dependencies that we're usingfor that, packaging those into a python package can help - one simple example