Thread
#prefect-community
    Josh

    Josh

    1 year ago
    Is it possible to run multiple flows from the same docker image? I’m using a docker agent and have multiple flows with the same set of dependencies. So I wanted all the flows to use the same docker image with different tasks and parameters per flow. When I run register the flows, each register call builds and pushes a docker image for that flow which will start to add up once I have more than a few flows. Is there a way to ask the flow/storage to check if a docker image exists already and only create if it doesn’t exist?
    Joël Luijmes

    Joël Luijmes

    1 year ago
    Yes this is possible! You can build the image with multiple flows, and then push it once. After which you can register the flows. I originally found this example https://gist.github.com/mousetree/8e58096ae3ad742407cd4042597984d6
    nicholas

    nicholas

    1 year ago
    Agreed with @Joël Luijmes here and I'd add that you could also use the ability to extend Docker images to build yourself a base image that includes those assets that change with less frequency and then extend that image with each of your flows.
    Josh

    Josh

    1 year ago
    thanks for the sample! I’m definitely going to build a base image with the dependencies and then a second layer image with the individual tasks I need.
    @Joël Luijmes How do you do this with run_configs. I saw that environments are deprecated and I’m getting this error
    Failed to load and execute Flow's environment: KeyError('<FlowName>')
    I’m trying to use docker storage on a vm if that helps.
    Was able to just use run_configs instead of environments and got it running