https://prefect.io logo
Powered by
Title
s

Sam Werbalowsky

01/13/2022, 9:59 PM
Anyone have issues with Git storage and a bunch of gobbedigook being appended to the repo? This is a new one, I’m fiarly sure that we haven’t updated anything on registration on our end: I checked by importing the same version of our helper library: 
<https://data:XXXXXXXXXXXX@https>://<my-repo>/data/scratch-data.git
It’s unexpected to see that 
<https://data>…@
portion
k

Kevin Kho

01/13/2022, 10:00 PM
That’s weird. First time I’ve seen it. How do you define your storage here?
s

Sam Werbalowsky

01/13/2022, 10:02 PM
This is part of a larger class…this is the relevant snippet: 
self.storage = Git(
                flow_path=flow_path,
                repo=self.project_path,
                repo_host="<https://mygitlabrepo.com>",
                git_token_secret_name="GITLAB_ACCESS_TOKEN",
                branch_name=self.branch_name,
            )
it’s hardcoded in what repo_host to use, which makes it stranger
edited because I am wondering if that is a gitlab ID for my company
k

Kevin Kho

01/13/2022, 10:06 PM
For this is the format Git storage uses. Anything before the @ is the secret. Maybe check the secret value?
flow.storage = Git(repo="my/repo", flow_path="/flows/flow.py", repo_host="<http://github.com|github.com>")
then you dont need 
https
for repo host
s

Sam Werbalowsky

01/13/2022, 10:07 PM
ah crap ok I see what happened thank you.
👍 1
yep, gotta rotate that out..is that a recent update to the Git storage? We didn’t have this issue before
k

Kevin Kho

01/13/2022, 10:16 PM
That file hasnt been edited since Oct 8 for 0.15.7
But doesnt seem like that was changed
Seems like it’s aways been like that
s

Sam Werbalowsky

01/13/2022, 11:07 PM
Weird - so on further investigation, it must be something in my Gitlab config or something…I made that update and I get that there is no repo at the location given..but what’s weird is I can run git clone with that location locally and clone it
k

Kevin Kho

01/13/2022, 11:10 PM
Do you self host your git repo?
s

Sam Werbalowsky

01/13/2022, 11:11 PM
yes
k

Kevin Kho

01/13/2022, 11:13 PM
I am wondering if there might be some authentication needed? We use this utility class to clone the repo if you want to test pulling directly
s

Sam Werbalowsky

01/13/2022, 11:16 PM
I wouldn’t think so only because we were registering/running flows before and this is a new problem with one specific repo. I just updated my helper library with the 
https
removed from the hostname and all worked fine.
the gitlab access token secret has worked in the past
k

Kevin Kho

01/13/2022, 11:18 PM
Ah ok. Are you good now?
s

Sam Werbalowsky

01/13/2022, 11:21 PM
No haha I’ll follow up tomorrow if I figure anything out
I guess what would help is if there’s anyway I can see what git command is being run to pull the repo from Git storage
This appears to be an issue with Dulwich v 0.20.30. I use poetry to manage my python packages and noticed that that version was used in the builds that were failing. I forced it back to version 0.20.26 and it worked fine. I am not sure if this should be considered a bug on Prefect or not, but I’m happy to put something in…I’m using v 0.15.10
k

Kevin Kho

01/14/2022, 3:41 PM
Let me check the requirements and setup and see if we need to pin
Yeah do you want to make a PR for it? https://github.com/PrefectHQ/prefect/blob/master/setup.py#L41
s

Sam Werbalowsky

01/14/2022, 3:47 PM
sure sounds good
k

Kevin Kho

01/14/2022, 3:54 PM
thank you!
s

Sam Werbalowsky

01/14/2022, 4:14 PM
https://github.com/PrefectHQ/prefect/pull/5326 - please let me know if anything needs to be added…as it is a simple change I did not add a file in to /changes but am happy to do so if required.
k

Kevin Kho

01/14/2022, 5:03 PM
You don’t want to be in our changelog? 😅
Ah I guess Michael said fixing compatibility would be better
s

Sam Werbalowsky

01/14/2022, 5:54 PM
hahaha fine by me, I just want my flows to work 😄
would’ve been smallest contribution ever probably
#prefect-serverJoin Slack
Powered by