https://prefect.io logo
#prefect-community
Title
# prefect-community
h

Henrietta Salonen

04/06/2022, 9:24 PM
Hi, I have two projects in Prefect Cloud: one for dev & one for prod. Each project has its own a Docker agent using DockerRun for running flows in containers (in EC2) built based on images from Dockerhub. These images are being re-built with a CI/CD flow (using a Dockerfile) whenever we push code to our Git repository either in dev or prod (one image tag for dev and one for prod) The flow source code is stored in Github and this is configured as the storage for our flows. For some reason my flow runs in the
prod
project suddenly fail with
Failed to load and execute flow run: ModuleNotFoundError("No module named 'project_name'")
however, all the flows with exactly the same source code in Prefect
dev
project run just fine. When I pull these images and start containers locally the project structure and packages are the same for both dev and prod. What could be the issue here?
k

Kyle McChesney

04/06/2022, 9:30 PM
I have run into somewhat similar issues. Do you have accessory/util code in your project as well, that the flow files import from?
upvote 1
a

Anna Geller

04/06/2022, 9:32 PM
I would guess the issue is that your GitHub storage assumes that some code from the Github repo is available even though it's not packaged into the Docker image
With
GitHub
storage, only the flow code gets pulled, not the whole repository. Can you share your code incl. your Dockerfile and your
storage
+
run_config
from your flow? without seeing the actual code you use, we can only make assumptions and point you in the right direction but it's hard to help without seeing your configuration
k

Kyle McChesney

04/06/2022, 9:40 PM
The fix for us was to ensure that we packaged our code as a proper python module that gets
python setup.py install
-ed into the docker image
h

Henrietta Salonen

04/06/2022, 9:40 PM
@Kyle McChesney yes, I do have some util code in my project as well and this is the part that I think is causing the issues
k

Kyle McChesney

04/06/2022, 9:40 PM
actual flow code is run from storage/flow submission, but the imports will still work if you install everything as a package.
upvote 1
h

Henrietta Salonen

04/06/2022, 9:41 PM
@Kyle McChesney I’ll try that one, thanks for the tip, still strange why it would work for one project but not for the other, right?
k

Kyle McChesney

04/06/2022, 9:41 PM
quite strange. I dont have any idea on that
h

Henrietta Salonen

04/06/2022, 9:42 PM
@Anna Geller I will DM you the configs and dockerfile
16 Views