https://prefect.io logo
Channels
Powered by
k

Karolína Bzdušek

02/18/2021, 5:43 PM
Please help, I am frustrated by this for too long. I am still chasing my tail 🙈 I want to have GitHub Storage, however I still can't make it work, although I am doing nothing fancy.
Copy code
flow.storage = GitHub(repo="cividi/gemeinde_workflows",path="/workflows/residential_density.py",access_token_secret="GITHUB_ACCESS_TOKEN")
"GITHUB_ACCESS_TOKEN"
is stored in secret in Cloud. However, when I run flow through UI, I am still getting this and not sure why. Note: repository is private and own by our team, I am using token generated under my account. What I am missing? Thanks for your help in advance.
z

Zanie

02/18/2021, 5:44 PM
Hi @Karolína Bzdušek -- did you mean to attach an error?
k

Karolína Bzdušek

02/18/2021, 5:45 PM
yes!
z

Zanie

02/18/2021, 5:52 PM
Hm, this is indeed peculiar. You're certain the token has permissions for that repository and that it's set in Cloud under that key?
k

Karolína Bzdušek

02/18/2021, 5:53 PM
yes, this was one of things I checked, I think I even tried to generate some under team's account, same error
z

Zanie

02/18/2021, 5:54 PM
Your 
flow.storage = Github(...)
code looks fine, it should work
Are you using a 
LocalAgent
?
Can you pass 
--env GITHUB_ACCESS_TOKEN=<your-token>
and see if that works?
💯 1
k

Karolína Bzdušek

02/18/2021, 5:55 PM
yes, that's why I am not happpy about it. And yes, 
LocalAgent
z

Zanie

02/18/2021, 5:56 PM
The auth logic looks like this:
Copy code
if self.access_token_secret is not None:
            # If access token secret specified, load it
            access_token = Secret(self.access_token_secret).get()
        else:
            # Otherwise, fallback to loading from local secret or environment variable
            access_token = prefect.context.get("secrets", {}).get("GITHUB_ACCESS_TOKEN")
            if access_token is None:
                access_token = os.getenv("GITHUB_ACCESS_TOKEN")
Should help us narrow down where the issue is
k

Karolína Bzdušek

02/18/2021, 5:57 PM
then this is really weird, it should load it from Secrets
I'll try the pass 
--env GITHUB_ACCESS_TOKEN=<your-token>
, I have to go to call now, write later.
Copy code
prefect agent local start  --env GITHUB_ACCESS_TOKEN=<token> --label <label1> --label <label2> --name "Local Agent"
And getting the very same log as before
z

Zanie

02/18/2021, 8:17 PM
Alright, thanks for checking that. One sec while I write up some code for a next step
Can you try the following with your token/repo?
Copy code
# Create a token at  <https://github.com/settings/tokens> with repo scope access
TOKEN = ""

# Set your repo name
REPO = "PrefectHQ/acsfas"

# Ensure you have python github installed
# - pip install PyGithub

from github import Github

client = Github(TOKEN)

client.get_repo(REPO)
You can use your existing token if you want, just want to ensure it works.
k

Karolína Bzdušek

02/18/2021, 8:28 PM
Copy code
Traceback (most recent call last):
  File "residential_density.py", line 178, in <module>
    client.get_repo(REPO)
  File "/home/karolina/gemeinde_workflows/prefect/lib64/python3.8/site-packages/github/MainClass.py", line 345, in get_repo
    headers, data = self.__requester.requestJsonAndCheck(
  File "/home/karolina/gemeinde_workflows/prefect/lib64/python3.8/site-packages/github/Requester.py", line 315, in requestJsonAndCheck
    return self.__check(
  File "/home/karolina/gemeinde_workflows/prefect/lib64/python3.8/site-packages/github/Requester.py", line 340, in __check
    raise self.__createException(status, responseHeaders, output)
github.GithubException.UnknownObjectException: 404 {"message": "Not Found", "documentation_url": "<https://docs.github.com/rest/reference/repos#get-a-repository>"}
z

Zanie

02/18/2021, 8:30 PM
So unfortunately there is an issue with your token or repo name, not with the secret/Prefect.
or a bug in the PyGithub library, but that seems less likely
k

Karolína Bzdušek

02/18/2021, 8:40 PM
Then I'll dig into that, because repo exists, I have copied the name and token should work as well.🤷‍♀️ Thank for helping me out and your time.
z

Zanie

02/18/2021, 8:43 PM
Sorry 😕 perhaps try just using 
curl
or something to hit the API (https://docs.github.com/en/rest/reference/repos#get-a-repository) to ensure the token is functioning as expected and eliminate more variables.
Powered by