https://prefect.io logo
m

madhav

05/06/2022, 7:11 PM
Hi - we are running into a problem with running a flow on a docker container. We get the following error:
Copy code
Failed to load and execute flow run: ModuleNotFoundError("No module named '/root/'")
Our setup: • a GitHub agent packages and deploys a docker container to ECR • ECS pulls the ECR image and deploys a fargate instance • (For our proof of concept) When the machine runs, we register a flow that lives in the image/fargate instance • After that we run
prefect local agent start
(both flow registration and starting the agent are run from a script that our ECS task definition runs on boot) • We do a “quick run” and get the module not found error. Other things to note: • all of our flows and supporting modules are on this container. • I’ve read some of the existing discussion surrounding this error and was not sure if the solution applied to our situation - given that we are running everything on the same Fargate instance
k

Kevin Kho

05/06/2022, 7:12 PM
I believe this is your issue. What is your Flow storage?
m

madhav

05/06/2022, 7:13 PM
Hi Kevin, thanks for the quick response! We are using local flow storage
k

Kevin Kho

05/06/2022, 7:14 PM
The problem is the container is looking for the Flow “locally” too inside the image.
Can I see the storage definition?
m

madhav

05/06/2022, 7:14 PM
Yea of course, the storage definition in the flow py file, right?
We don’t have any storage definitions in our flow file, it uses the default storage defs. Could that be the problem?
k

Kevin Kho

05/06/2022, 7:16 PM
Yeah that is the problem because you want to point it to the location of the Flow inside the container
m

madhav

05/06/2022, 7:17 PM
Got it
This is a good example
m

madhav

05/06/2022, 7:18 PM
fantastic, thank you Kevin
👍 1
m

Malthe Karbo

05/07/2022, 10:03 AM
If you pack the entire environment incl. package and build venv / install inside docker image, we had great success using module storage (it will import using python import strings, e.g.,
mypackage.somepath.flow
which is valid, since the agent had everything installed in .venv).
44 Views