Another thing is that I am getting a strange error...
# prefect-server
j
Another thing is that I am getting a strange error for this simple Flow:
Copy code
Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'C'")
Can anybody tell me some hint where to look for a cause? googling did not help... 😞 More details in thread.
g
Would you mind please moving all the lengthy stacktrace etc. into this thread, rather than being in the main post? This makes it more comfortable for those browsing the slack channel.
j
yep, sorry for that 😐 UI error:
Copy code
Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'C'")
Copy code
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/prefect/__main__.py", line 4, in <module>
    cli()
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/prefect/cli/execute.py", line 90, in flow_run
    raise exc
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/prefect/cli/execute.py", line 67, in flow_run
    flow = storage.get_flow(flow_data.name)
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/prefect/storage/local.py", line 104, in get_flow
    return extract_flow_from_module(module_str=flow_location)
  File "/home/jakub_vedral/prefect-env/lib/python3.7/site-packages/prefect/utilities/storage.py", line 131, in extract_flow_from_module
    module = importlib.import_module(module_name)
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'C'
flow definition:
Copy code
from datetime import timedelta
from prefect import Flow
from prefect.schedules import IntervalSchedule
from prefect import task

@task(name="test")
def test():
    return "Hi!"



with Flow("Test Run",IntervalSchedule(interval=timedelta(hours=4)),validate =True) as f:

    test()



f.register(project_name="test",labels=["production"])
Prefect server is running on my GCP instance under virtualenv  -  Linux prefect-prod 4.19.0-13-cloud-amd64 #1 SMP Debian 4.19.160-2 (2020-11-28) x86_64 pip packages of server instancne:
Copy code
cached-property==1.5.2
certifi==2020.12.5
cffi==1.14.4
chardet==4.0.0
click==7.1.2
cloudpickle==1.6.0
croniter==0.3.37
dask==2021.1.1
distributed==2021.1.1
docker==4.4.1
HeapDict==1.0.1
idna==2.10
importlib-resources==5.1.0
marshmallow==3.10.0
marshmallow-oneofschema==2.1.0
msgpack==1.0.2
mypy-extensions==0.4.3
natsort==7.1.1
pendulum==2.1.2
pkg-resources==0.0.0
prefect==0.14.5
psutil==5.8.0
pycparser==2.20
pygit2==1.5.0
python-box==5.2.0
python-dateutil==2.8.1
python-slugify==4.0.1
pytz==2020.5
pytzdata==2020.1
PyYAML==5.4.1
requests==2.25.1
six==1.15.0
sortedcontainers==2.3.0
tabulate==0.8.7
tblib==1.7.0
text-unidecode==1.3
toml==0.10.2
toolz==0.11.1
tornado==6.1
urllib3==1.26.3
websocket-client==0.57.0
zict==2.0.0
zipp==3.4.0
z
Hi @Jakub Vedral — I don’t see how this error could be thrown from the code you provided. It’s entering a code path that I’m pretty sure is only encountered when
Local
storage is explicitly defined.
Are you sure this is the same flow code that’s erroring?
j
Yes, I am sure. I think it is related to the server environment and running prefect server from venv, but i dont know how to debug that. When i run it locally from my windows machine, it runs fine.
z
Can you try adding
flow.storage = Local()
and reregistering?
It’s trying to import it from a module which should not be the default behavior — I’m not even sure where it’s getting a module path from
Can you also provide the output of
prefect diagnostics
on the machine where you are registering the flow?
j
prefect diagnostics: { "config_overrides": { "context": { "secrets": false } }, "env_vars": [], "system_information": { "platform": "Linux-4.19.0-13-cloud-amd64-x86_64-with-debian-10.7", "prefect_backend": "server", "prefect_version": "0.14.5", "python_version": "3.7.3" } }
setting flow.storage = Local() did not help, still the same error
z
Can you print
flow.storage.path
?
Nevermind that’s not what we want
So this module-not-found error indicates that the flow path is
C:\….
which is a windows path and it’s not found on the file system where your agent is running so it tries to load it from a module
Have you registered this flow on a windows machine?
j
yep i have a debian linux machine as a server on GCP and registered the flow from my windows machine
z
Ah okay, and where are you running your agent?
j
my agent is also on the linux server
it has the same pip packages as my local windows machine
z
Alrighty — so the issue here is that your flow is using local storage because you aren’t setting a storage so it’s being written to your windows machine’s file system then your agent can’t find it because it’s running on your server.
Generally this is avoided by setting labels on flows that are local to the hostname of the system that has the flow file on it so the agent won’t try to pick it up but it looks like you’re setting labels explicitly.
j
yep, i realized that if i dont make the labels of agent and my flow the same it will not start. I can see now why is that implicitly taken care of 🙂
z
You’ll need to switch to some form of remote storage so the agent can access the flow and you’ll be good!
j
Well I changed the storage to the remote and everything works. Thanks so much for your help 🙂