Greg Roche09/22/2020, 2:35 PM
|--project |-.venv |-etl |--__init__.py |--foo | |--__init__.py | |--foo_flow.py |--bar | |--__init__.py | |--bar_flow.py |--shared_utils |--__init__.py |--utils.py
I import modules from
by appending the etl folder to sys.path before importing
. When I run any of these flows with
they work fine. However, when I do
instead and then start a local agent, from any of the directories listed above, all flow runs initiated from the server to the agent fail instantly with
. According to a stackoverflow answer this is because the agent's python path doesn't include
Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'shared_utils'"
but it still fails when I run the agent with
. I've tried registering the flow from every directory listed above, and tried starting the agent from every directory listed above, and also tried passing every directory listed above as an
argument, and I get exactly the same error every time. Can anybody please point out what I'm missing?
is not the correct name of the CLI flag, it’s actually
, or the more abbreviated
Let me know if that solves it for you!
Greg Roche09/22/2020, 3:58 PM
in my script. After a bunch more testing, I've found a combination of options that doesn't cause the flow to error out immediately: both registering the flow and starting the agent from
now just makes the flow hang forever with status "running flow" in the server UI. If I run the agent with
I see a torrent of errors mentioning
before the following error repeats endlessly over and over until I kill the agent:
ModuleNotFoundError: No module named 'setuptools._distutils'
tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: None, threads: 4>> Traceback (most recent call last): File "c:\project\.venv\lib\site-packages\tornado\ioloop.py", line 907, in _run return self.callback() File "c:\project\.venv\lib\site-packages\distributed\nanny.py", line 414, in memory_monitor process = self.process.process AttributeError: 'NoneType' object has no attribute 'process'
Greg Roche09/22/2020, 4:04 PM
simone12/19/2020, 11:17 AM
from the package directory works but if i add any
prefect agent local start --show-flow-logs
option that point to the directory of the package (even if i am inside the package) I get the
Greg Roche12/24/2020, 8:27 AM
. There's another thread here https://prefect-community.slack.com/archives/CL09KU1K7/p1605089116281400 where a Prefect dev provided me with an example project structure to get me started, that was super helpful and will probably point you in the right direction too.
pip install -e .
simone01/04/2021, 7:07 AM