Hey Prefect community! I’m using ECS task runs wit...
# prefect-community
Hey Prefect community! I’m using ECS task runs with Github storage for my flows but running into the following error when running the task:
Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'constants'")
I’m importing constants which will be shared across many flows: the project name, ECS cluster name, Github Token secret name from a
file in the same directory. Running the flow locally with
works just fine as expected, but when running it in ECS
is not found. How do I register other module dependencies with Github storage, if possible? Or do I have to add these constant values to each of my flow files? Thank you! Example below:
Copy code
. (repo root)
| |____hello.py
| |____constants.py
| |______init__.py
Are you on Prefect Cloud? This way maybe you could store those constants in KV Store instead. The issue is that Github storage can only store your flow, but not the flow's dependencies and utilities such as your constants.py file. You would need to either build and install it as a package or change the strategy how you generate those constants e.g. moving to KV Store/AWS Parameter store or using Prefect Parameter tasks instead... just some ideas this thread also discussed this problem
upvote 1
an alternative would be to use Docker storage to package both flow and its code dependencies into a single Docker image that you can push to ECR - this way you would have one image per flow
Yes we’re on Prefect Cloud - and that makes sense. Will evaluate both those options! Thanks for the thread link and context on the dependencies being separate from the flow 🙂
👍 1
Hey Wesley, could you move the code in this thread when you get the chance so we can keep the main channel a bit more compact?
👍 1