Hi everyone, I’m having trouble switching from Docker storage (which runs successfully) to Github storage.

flow.storage = GitHub(secrets=["GITHUB_ACCESS_TOKEN"],
                      repo="my_org/repo_name",
                      path="prefect_flows/flows/flow.py",
                      ref="trunk")

The access token is saved in prefect cloud under the same name. During the flow run prefect can’t seem to be able to find the repo??

INFO:Downloading flow from GitHub storage - repo: 'my_org/repo_name', path: 'prefect_flows/flows/flow.py', ref: 'trunk'

ERROR: Repo 'my_org/repo_name' not found. Check that it exists (and is spelled correctly), and that you have configured the proper credentials for accessing it.

ERROR: Failed to load and execute Flow's environment: UnknownObjectException(404, {'message': 'Not Found', 'documentation_url': '<https://docs.github.com/rest/reference/repos#get-a-repository>'}, {'server': '<http://GitHub.com|GitHub.com>', 'date': 'Fri, 11 Jun 2021 22:46:06 GMT', 'content-type': 'application/json; charset=utf-8', 'x-github-media-type': 'github.v3; format=json', 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset', 'access-control-allow-origin': '*', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'x-frame-options': 'deny', 'x-content-type-options': 'nosniff', 'x-xss-protection': '0', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'content-security-policy': "default-src 'none'", 'vary': 'Accept-Encoding, Accept, X-Requested-With', 'content-encoding': 'gzip', 'x-ratelimit-limit': '60', 'x-ratelimit-remaining': '59', 'x-ratelimit-reset': '1623455165', 'x-ratelimit-resource': 'core', 'x-ratelimit-used': '1', 'content-length': '112', 'x-github-request-id': 'C202:57D9:E1FC5:1D69CC:60C3E7AD'})

I’ve curled the github api endpoint

repos/

directly using the same auth token and I do get the repo’s information. Pls advise! 🙏

Hey @Fina Silva-Santisteban, looking at the docs , I think

secrets

might have to be

access_token_secret="GITHUB_ACCESS_TOKEN"

@Kevin Kho thanks so much, that fixed!! Using

secrets

wasn’t throwing any errors since the parent class of the github storage has that attribute. I wonder why

access_token_secret

is an optional parameter for Github storage initialization. Don’t you always need it to access the repo? 🤔

No if the repo is public. I think it can pull it directly

@Kevin Kho I actually have a different problem now, related to this thread^. I was trying to debug it myself first, unfortunately unsuccessfully so here I am again. 😅 Ok, so as mentioned last week your suggestion fixed the github permission problem (thank you for that!), but the error I get now is that it’s not able to execute the flow’s environment (see screenshot). The folder

prefect_flows

lives in the root project folder, so it’s strange that it wouldn’t be able to find it.

I had someone with this issue today. I made a working example . I think it might need a leading ‘/’

@Kevin Kho Thank you for the link to your demos! (All of those demos are really helpful in general, the docs should include them!) That’s the thing, I’m importing other files in my flow. A flow lives inside

prefect_flows/flows/

and the tasks it uses live inside

prefect_flows/tasks/

, so the first statement in my flow file is an import

from prefect_flows.tasks.task_a import do_a()

. I think that’s what the error is complaining about, right? Does this mean github storage only works for flows that hold their entire business logic in that same file?

Thanks for the comment! Yes exactly that’s the error. Github Storage only keeps the files. If you need to keep dependencies, you need to package it as a Python library and then install that in the Docker image. I have another example of that in the demo repo….lol

@Kevin Kho Got it! We’ve been using docker storage and that works fine, I don’t even have to package it as python library, regular import statements are enough. I was hoping to be able to switch to github storage, but it looks like we need to stick to docker storage!

Oh I see, yeah only Docker storage can hold dependencies but loading Git repos as modules is on the roadmap.

@Kevin Kho great! Thanks so much for all of your help again! 🙏