Hello, is there a recommended approach for making a local Python module accessible to multiple docker containers? I’m running on ECS Fargate with flows stored in S3. I had hoped the serialization process would have pulled out the required objects from the import statement at build time but that doesn’t appear to be the case. Is it better practice to COPY the module in each container’s Dockerfile instead?

you’re 100% correct, that’s the way to do this. Let me share an example

@Anna Geller (old account) thanks! I’ll try that approach

the easiest way would be to include the setup.py and install the package within your docker image e.g. https://github.com/anna-geller/packaging-prefect-flows/blob/master/Dockerfile#L7 maybe this post can help: https://medium.com/the-prefect-blog/the-simple-guide-to-productionizing-data-workflows-with-docker-31a5aae67c0a Otherwise, if you still face any issues, feel free to share your Dockerfile and flow definition so that we can have a look

@Anna Geller (old account) thanks for the links. Do you know where an unpickled flow would run from on the container? I’d rather keep this as a module but I can convert it to a package if there isn’t a way to make that import work.

I think that you shouldn’t worry about where the flow gets stored when you use pickle storage. In contrast, when you add

stored_as_script=True

within your Docker storage, then it does matter where you put the flow file. I think those two examples clarify that: 1. Stored as script - you don’t build the image on registration https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows_no_build/docker_script_docker_run_local_image.py 2. Flow stored as pickle - you do build the image during registration https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows/docker_pickle_docker_run_local_image.py