https://prefect.io logo
Title
k

Kelly Huang

03/07/2022, 11:27 PM
Hi, I'm trying to run flows on the cloud using GitHub storage and a local agent. But I'm getting this error:
in get_flow
  from github import UnknownObjectException
ModuleNotFoundError: No module named 'github'
I'm reading differing things about whether or not github storage can access package dependencies? Correct me if I'm wrong, but shouldn't it definitely be able to access my Pipfile and therefore dependencies? Otherwise what would the point of github storage be?
:discourse: 1
Follow up question: It may be that I'm just not configuring the github repo correctly. I created a github access token on my personal account, but I want to access a repo in a shared workspace of the organization I'm part of. Is that possible? And if so, is this correct:
repo="workspace/repo" not "kellyhuang/repo"
does that make sense?
k

Kevin Kho

03/08/2022, 12:35 AM
Hi @Kelly Huang, is
github
a custom module for you? The answer here is no though that GitHub storage cannot store custom modules because we do pull the repo, but the path resolving is really hard if not impossible to make those modules importable. It might be possible in the near future with changes to cloudpickle that allow deep serialization of modules, but this would be an Orion think as opposed to current Prefect. If you are using a local agent though, you can run point to the working dir you want to run the flow in using LocalRun. You can also start the agent out of an environment and that will use that environment to run Flows.
On the token, I think you should have access to workspace repos yep
k

Kelly Huang

03/08/2022, 6:22 PM
@Kevin Kho Hmm no it's not a custom module, which is why I'm confused:
File "/Users/kellyhuang/.local/share/virtualenvs/etl-_MbHxiVk/lib/python3.8/site-packages/prefect/storage/github.py", line 62, in get_flow
    from github import UnknownObjectException
k

Kevin Kho

03/08/2022, 6:23 PM
Ah then you might just need the Github extra?
pip install prefect[github]
?
đź‘€ 1
k

Kelly Huang

03/08/2022, 6:27 PM
@Kevin Kho do you know how to pipenv install that? I tried
pipenv install --pre prefect[github]
which failed
k

Kevin Kho

03/08/2022, 6:28 PM
What is your error? Why do you use the
--pre
flag?
k

Kelly Huang

03/08/2022, 6:38 PM
no matches found: prefect[github]
k

Kevin Kho

03/08/2022, 7:00 PM
Try “prefect[github]” with the quotes