k

    Kelly Huang

    6 months ago
    Hi, I'm trying to run flows on the cloud using GitHub storage and a local agent. But I'm getting this error:
    in get_flow
      from github import UnknownObjectException
    ModuleNotFoundError: No module named 'github'
    I'm reading differing things about whether or not github storage can access package dependencies? Correct me if I'm wrong, but shouldn't it definitely be able to access my Pipfile and therefore dependencies? Otherwise what would the point of github storage be?
    Follow up question: It may be that I'm just not configuring the github repo correctly. I created a github access token on my personal account, but I want to access a repo in a shared workspace of the organization I'm part of. Is that possible? And if so, is this correct:
    repo="workspace/repo" not "kellyhuang/repo"
    does that make sense?
    Kevin Kho

    Kevin Kho

    6 months ago
    Hi @Kelly Huang, is
    github
    a custom module for you? The answer here is no though that GitHub storage cannot store custom modules because we do pull the repo, but the path resolving is really hard if not impossible to make those modules importable. It might be possible in the near future with changes to cloudpickle that allow deep serialization of modules, but this would be an Orion think as opposed to current Prefect. If you are using a local agent though, you can run point to the working dir you want to run the flow in using LocalRun. You can also start the agent out of an environment and that will use that environment to run Flows.
    On the token, I think you should have access to workspace repos yep
    k

    Kelly Huang

    6 months ago
    @Kevin Kho Hmm no it's not a custom module, which is why I'm confused:
    File "/Users/kellyhuang/.local/share/virtualenvs/etl-_MbHxiVk/lib/python3.8/site-packages/prefect/storage/github.py", line 62, in get_flow
        from github import UnknownObjectException
    Kevin Kho

    Kevin Kho

    6 months ago
    Ah then you might just need the Github extra?
    pip install prefect[github]
    ?
    k

    Kelly Huang

    6 months ago
    @Kevin Kho do you know how to pipenv install that? I tried
    pipenv install --pre prefect[github]
    which failed
    Kevin Kho

    Kevin Kho

    6 months ago
    What is your error? Why do you use the
    --pre
    flag?
    k

    Kelly Huang

    6 months ago
    no matches found: prefect[github]
    Kevin Kho

    Kevin Kho

    6 months ago
    Try “prefect[github]” with the quotes