Hi Team, I am currently facing an issue where my flow references a module within the same repository but is unable to load the module at execution time. I am using Prefect Cloud, GH Storage, and a Kubernetes Agent. Seems it could potentially be a pathing issue as I must set
PYTHONPATH
in my local environment, but not having much luck setting a similar ENV variable on the agent or job_spec. Any suggestions or documentation that might help get over this hump?
Repository Structure:
Copy code
/src
--/flows
----/flow.py
--/tasks
----/task.py
Error Message:
Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'src.tasks'")
n
nicholas
03/19/2021, 6:38 PM
Hi @Zach Hodowanec - when using GitHub storage, only the file for the flow location is pulled, not the entire repo. I'd recommend explicitly declaring those modules and transitioning to Docker storage.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.