Hello, I hope this is the right place to ask for help but if not please let me know. I've been working on getting Prefect Cloud set up on AWS and things are going quite well. I've hit a snag that I can't seem to resolve. I'm getting
Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'xxx'")
when submitting the flow to and ECS(Fargate). The module in question is a custom Python module that I include in the agent Docker image and it's installed via pip install. I feel like I'm missing something here but from what I've read, that should work. Does anyone have a tip on how I might get the flow execution to be aware of this custom module? Is the Agent the proper place to install the module?
k
Kevin Kho
12/08/2021, 7:32 PM
Hey @Brian S, this is the right place. The agent is not the right place. You attempt is good. Can you show me your Dockerfile? Can you download the image and exec into it and start an interaction python shell then try to import it? Try in different directories to see if installation was right.
Kevin Kho
12/08/2021, 7:33 PM
I have a guide here . May help or you may be past this. Would be good for double checking though
b
Brian S
12/08/2021, 7:33 PM
awesome, let me check the guide and make sure I'm doing this correctly
Brian S
12/08/2021, 7:34 PM
You mention the Agent container isn't the correct place for that so I'm pretty sure that's my first problem hehe
k
Kevin Kho
12/08/2021, 7:35 PM
Well because ECS execution environment is different
b
Brian S
12/08/2021, 7:38 PM
I see the err of my ways. This document seems to have the answer.
t
Tilak Maddy
12/08/2021, 8:38 PM
@Brian S Add this line to your docker file (if you are not using setup.py)
ENV PYTHONPATH "${PYTHONPATH}:/app"
@Kevin Kho is really trying to hide the magic line that fixed my deployment issue . (I had the same error)
k
Kevin Kho
12/08/2021, 8:45 PM
pip install should be sufficient though
b
Brian S
12/10/2021, 8:20 PM
Thanks so much guys! I'm pretty much over the hump thanks to you
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.