Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hey, another question regarding using ECS Fargate and S3 for deploying flows :slightly_smiling_face: I have a common python file with many helper methods that are used by all of my different flows, however, they are obviously not deployed together with the flow themselves when using
```flow.register(project_name="test-project")```
What is the best practice to importing common .py files? Since at the moment the simple
```from utils import util```
fails with an error:
```Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'utils'")```

The best way would be to build a custom Docker image that has those modules and push it to ECR - you can then reference this image in your ECSRun

In the same tutorial you used for the agent, there is a section called “Build a custom Docker image for your Prefect flows”. You can use e.g. this repository and <https://github.com/anna-geller/packaging-prefect-flows/blob/master/Dockerfile|Dockerfile> as a template for your image.

^ this is what we do at my org <@U02N6AC6VB6>; we have a couple of base images that have all of our org's custom boilerplate modules + utilities that we need for running any of our flows in our remote environments, it's been working nicely for us thus far

actually, <@U02H1A95XDW> if I could tack onto this thread, I have a somewhat related question. At my org we have these "base images" that don't have any additional package requirements on top of what's already shipped with the official prefect image, and from there when developers need images that have specific package dependencies, we have them use one of our base images in their Dockerfile's `FROM` clause, and then install their additional dependencies via a `requirements.txt` after that.

I know this is a common issue in Python development, but how does Prefect recommend go about managing differing base dependencies between prefect and additional packages that a developer is trying to introduce? e.g. I want to install the latest version of `paramiko`, which by default uses newer versions of `cryptography`, `pynacl`, etc. than the version of prefect I'm running does. Right now, we're just blindly installing these additional modules, and if things work out and there are no conflicts, then great; but I'm certain there will come a day when there are _actually_ some conflicting dependencies, and we'll probably need to introduce some kind of pre-hook at uses something like `pip-tools` to install newer dependencies while constraining some installed package versions (based on what's already installed in the official prefect images)

Is this something that the engineers at Prefect see enough to have some opinions on how to go about managing it? I know this is really a much more general question and has nothing to do with prefect per se, but I've had a lot of luck getting tips from Prefect support + the community here so I figured it's worth a shot asking here :sunglasses:

If you’re asking about Prefect recommendation, then there is only one: using Prefect’s base images for your base images. Everything else is really up to you.

Regarding conflicts - it’s an inevitable reality of Python that a conflict may happen if a package you are using has a different requirements than your script/application does. You could try disabling transitive dependency updates e.g. by passing `--no-update-dependencies` or `--no-update-deps` to `conda install` command. Ex: `conda install --no-update-deps paramiko` .

In pip it would be:
```pip install --no-deps paramiko```
