https://prefect.io logo
j

Jan Marais

02/03/2021, 1:21 PM
Is it possible to use S3 storage if your flow depends on code in external modules? I keep running into this error:
Copy code
Failed to load and execute Flow's environment: StorageError('An error occurred while unpickling the flow:\n  ModuleNotFoundError("No module named \'project\'")\nThis may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')
Using ECSAgent and a custom image in ECR. I have no problem importing from the external module inside the container or in the environment where my agent is launched or where my flow is registered from.
j

josh

02/03/2021, 1:26 PM
Hi @Jan Marais yes it is possible! When you say you have no problem importing the external module inside the container that means you are able to go into your container and call
import project
and it works?
j

Jan Marais

02/03/2021, 1:26 PM
yes
j

josh

02/03/2021, 1:27 PM
Hm interesting because that should mean it would work. Would you be able to open this as an issue in the repo with more information on your flow/storage/etc. to better diagnose?
j

Jan Marais

02/03/2021, 1:27 PM
another strange thing, when I just have a
import project
line in the script registering the flow, the flow runs successfully, but when I reference the
project.some_function
in a task, I get the above error
Okay, will create an issue
j

josh

02/03/2021, 1:28 PM
Are you able to access the
project.some_function
manually when you run the container?
j

Jan Marais

02/03/2021, 1:28 PM
yes
j

josh

02/03/2021, 1:28 PM
Weird 🤔
@josh there was a mismatch in the version of the image I pulled for the flow vs the one I was working on 🙈
🎉 1
Should I close or delete the issue?
👍 1