https://prefect.io logo
m

Mitchell Bregman

10/22/2020, 7:54 PM
How can I register a flow from the command line? My setup:
Copy code
# $ROOT/src/flow.py
from prefect import Flow

flow = Flow(name="test flow")
Now from the command line in dir
$ROOT
, I run:
prefect register flow --file src/flow.py --name "test flow"
I get the error
Copy code
KeyError: "'__name__' not in globals"
What am I doing wrong? I’d like to use the CLI as it will be more simplistic from a CI/CD standpoint
d

Dylan

10/22/2020, 8:44 PM
Hi @Mitchell Bregman that looks correct to me 🧐
Only thing I see is that there are no quotes for the name in the doc https://docs.prefect.io/api/latest/cli/register.html#register
Sometimes that can be finicky
m

Mitchell Bregman

10/22/2020, 8:45 PM
Should I replace the spaces with
-
?
d

Dylan

10/22/2020, 8:45 PM
couldn’t hurt
m

Mitchell Bregman

10/22/2020, 8:45 PM
lets see! trying now
z

Zanie

10/22/2020, 8:46 PM
If that doesn’t work, could you post the full traceback for the error?
m

Mitchell Bregman

10/22/2020, 8:47 PM
Copy code
Traceback (most recent call last):
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/bin/prefect", line 8, in <module>
    sys.exit(cli())
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/prefect/cli/register.py", line 86, in flow
    flow = extract_flow_from_file(file_path=file_path, flow_name=name)
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/prefect/utilities/storage.py", line 76, in extract_flow_from_file
    exec(contents, exec_vals)
  File "<string>", line 18, in <module>
KeyError: "'__name__' not in globals"
didnt work
z

Zanie

10/22/2020, 8:57 PM
Just to check, did you share your whole whole
flow.py
file with us? Sometimes this happens when you do relative imports within a script.
m

Mitchell Bregman

10/22/2020, 8:59 PM
I didnt share the full
flow.py
.. just to preface, i set up a
cookiecutter
project for my team so that we can have the similar CI/deployment recipe, etc. this is based on that cookiecutter. ill push to GH so you can see the resulting project, 1 sec
z

Zanie

10/22/2020, 9:11 PM
So I presume this is because in
__init__.py
you do a relative import from
.flow
- https://github.com/mitchbregs/cookiecutter-prefect/blob/master/%7B%7Bcookiecutter.service_name%7D%7D/src/__init__.py#L2
This could just be
from src.flow import MyFlow
I actually couldn’t reproduce the error you got after cloning your code locally. I fixed that import path and then added
MyFlow = flow
to your
flow.py
file and did
pip install -e .
and things seem to have worked
m

Mitchell Bregman

10/22/2020, 9:43 PM
sorry, had to step away
will get back to you in a few!
re: 
MyFlow = flow
 where did you add that? and whats the reasoning?
z

Zanie

10/22/2020, 10:03 PM
You have
from .flow import MyFlow
but in
flow.py
the name of the flow is just
flow
not
MyFlow
So the import or the name of the flow variable has to be changed and I choose to duplicate the flow variable to avoid making more changes to your code
m

Mitchell Bregman

10/22/2020, 10:05 PM
i must have pushed up a diff branch to GH… 1 sec, will send u the project generated
so the way this is setup ^ when i
prefect register
i get the following traceback
Copy code
Traceback (most recent call last):
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/bin/prefect", line 8, in <module>
    sys.exit(cli())
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/prefect/cli/register.py", line 86, in flow
    flow = extract_flow_from_file(file_path=file_path, flow_name=name)
  File "/Users/mitchell.bregman/git/pmflows_oj0hxdhv-my_first_flow/venv/lib/python3.8/site-packages/prefect/utilities/storage.py", line 76, in extract_flow_from_file
    exec(contents, exec_vals)
  File "<string>", line 18, in <module>
ModuleNotFoundError: No module named 'src'
i have to change all the imports to relative imports in order to get the
__name__
not in globals arg
some weird disconnect is happening
i.e., this https://github.com/mitchbregs/example-prefect/blob/master/src/flow.py#L18-L19 needs to change to
Copy code
from . import config
from .tasks.task import MyTask
once i do that, then i get the
__name__
error
i tried removing the
__init__
too
z

Zanie

10/22/2020, 10:14 PM
ModuleNotFoundError: No module named 'src'
means you have not installed your module which you would do with
pip install -e .
from the top-level where your setup.py is
m

Mitchell Bregman

10/22/2020, 10:14 PM
wow
im such a moron lol
thank you so so much!
that solved everything
z

Zanie

10/22/2020, 10:15 PM
Using relative imports will cause the
name
error because of how python imports work — we are executing your code without it knowing what file its in
Wonderful!
Glad we got it sorted out 🙂
Sweet cookie cutter project though, looks like a great idea.
m

Mitchell Bregman

10/22/2020, 10:15 PM
heck yeah! just trying to streamline my teams development processs…. not everyone cares about handling ci/deployment, etc…. so i figured this would solve a lot of the annoying parts
thank you!!!
i plan to write about it sometime soon!
👍 1
z

Zanie

10/22/2020, 10:17 PM
Looking forward to it! Post it here for us 🙂
💯 2