https://prefect.io logo
j

Josh

01/15/2021, 5:38 PM
Is it possible to run multiple flows from the same docker image? I’m using a docker agent and have multiple flows with the same set of dependencies. So I wanted all the flows to use the same docker image with different tasks and parameters per flow. When I run register the flows, each register call builds and pushes a docker image for that flow which will start to add up once I have more than a few flows. Is there a way to ask the flow/storage to check if a docker image exists already and only create if it doesn’t exist?
j

Joël Luijmes

01/15/2021, 6:29 PM
Yes this is possible! You can build the image with multiple flows, and then push it once. After which you can register the flows. I originally found this example https://gist.github.com/mousetree/8e58096ae3ad742407cd4042597984d6
🙏 1
upvote 1
n

nicholas

01/15/2021, 9:36 PM
Agreed with @Joël Luijmes here and I'd add that you could also use the ability to extend Docker images to build yourself a base image that includes those assets that change with less frequency and then extend that image with each of your flows.
💯 1
j

Josh

01/15/2021, 10:10 PM
thanks for the sample! I’m definitely going to build a base image with the dependencies and then a second layer image with the individual tasks I need.
@Joël Luijmes How do you do this with run_configs. I saw that environments are deprecated and I’m getting this error
Failed to load and execute Flow's environment: KeyError('<FlowName>')
I’m trying to use docker storage on a vm if that helps.
Was able to just use run_configs instead of environments and got it running
🚀 2
4 Views