https://prefect.io logo
Title
f

Florian Guily

05/23/2022, 10:25 AM
Hey, i'm having a hard time to understand how to provide a branch name to the github task. The doc is mentionning the
ref
parameter but it is refering to a "SHA-1 value, tag, or branch name". Where can i find this SHA1 value of a given branchname ?
e

emre

05/23/2022, 10:50 AM
The
commit-sha
,
tag
and
branch name
should all be interchangeable as far as github api is concerned. Just give the name of the branch and it should resolve successfully to the underlying commit sha. To find the commit sha, run
git log
while you are checked out to your branch. The entry at the top is yout commit sha. Should have something like
(HEAD -> your_branch_name)
next to it, to identify that your branch currently refers to that specific commit.
🔝 1
f

Florian Guily

05/23/2022, 10:53 AM
i already tried to provide the branch name but it doesn't seems to work. Maybe i'm doing it wrong. I have this error message
"No commit found for SHA: {'dev'}"
e

emre

05/23/2022, 10:55 AM
which github task are we talking about 😅
f

Florian Guily

05/23/2022, 10:56 AM
oh yeah sorry, i'm talking about the storage one
to pull flow code from a github repo if i understood correctly
e

emre

05/23/2022, 11:12 AM
weird, I can run the part where the branch name is used.
f

Florian Guily

05/23/2022, 11:13 AM
must come from me then... You just provide the branchname as a string ?
e

emre

05/23/2022, 11:13 AM
are you somehow passing
'dev'
within a python set?, the error including curly braces seems weird.
f

Florian Guily

05/23/2022, 11:14 AM
yeah i find it weird too. I'm passing it like this:
BRANCH = "dev"

with Flow(FLOW_NAME,
        storage=set_storage(BRANCH),
        run_config=set_run_config()) as flow:
e

emre

05/23/2022, 11:14 AM
yeah:
gg = GitHub(repo="PrefectHQ/prefect", path="/", ref="master")

with Flow("abcdef", storage=gg) as f:
    ab()

gg.add_flow(f)

gg.get_flow("abcdef")
f

Florian Guily

05/23/2022, 11:15 AM
with set_storage being:
def set_storage(branch: str) -> GitHub:
    return GitHub(
        repo=f"my/repo",
        path=f"my/path.py",
        ref=branch,
        access_token_secret="GITHUB_ACCESS_TOKEN",
    )
e

emre

05/23/2022, 11:16 AM
code seems correct, I can think of two issues: 1- dev really doesn't exist 😅 2- your access token isn't authorized to read dev
🔝 1
f

Florian Guily

05/23/2022, 12:22 PM
yeah i thought about issue 2 but i did add permissions to read and write org repo/project (this is an org repo)
e

emre

05/23/2022, 12:32 PM
your
set_storage
call receives 2 inputs, but the implementation expects one. Probably happened while you are changing your code for slack, but maybe we lost the erroneous bit in translation? could you double check?
f

Florian Guily

05/23/2022, 12:35 PM
yes sorry for slack purpose i deleted the second param. I edited the call to match
e

emre

05/23/2022, 12:41 PM
yeah im stumped, im pretty sure github storage works with branch names though
f

Florian Guily

05/23/2022, 12:44 PM
yeah no worries, i'll see with the org owner because there are authorization for the personnal access token to get if the org uses saml authentication.
Quick update, it seems the issue is that the code from set_storage() is somehow not updated when i register the flow and it keeps using the first version which was providing
{"branch"}
as a ref (i understand the error now). Despite changing the code, it seems that it keep executing a "cached" version of this function and i can't understand why
i'm a bit struggling understanding which version of the code is executed after submitting a flow, especially when adding this "storage" dimension
i have the same architecture as @Anna Geller github repo used for tutorial purpose https://github.com/anna-geller/prefect-dbt-k8s-snowflake
any idea about this "caching" thing @Anna Geller?
I found the issue, the problem was coming from the fact that my set_storage function was in an other folder and was imported in the main flow file as a module. Because it is done that way, the module version is determined by the status.py execution. I just had to re execute
pip install .
to update the changes and register the flow again to finally have an up to date
set_storage
!
🤯 1
a

Anna Geller

05/23/2022, 2:39 PM
that makes sense, thanks so much for the update
in the future, to avoid such issues while you are still in a development mode, you can install your flow_utilities in editable mode so that all changes you keep making are reflected automatically:
pip install -e .
f

Florian Guily

05/23/2022, 2:58 PM
ooh i didn't know about that ! thanks
👍 1