I am using

prefect.yaml

to configure a

git_clone

pull step that uses gitlab-credentials to clone a private repo. It seems that the token which is of type pydantic SecretStr isn’t being resolved before the command of

git clone …

is run. Has anyone else encountered this?

hey @Mattijs De Paepe - can you show your

pull

step?

pull:
- prefect.deployments.steps.git_clone:
    repository: <https://git_url/some_group/repo.git>
    branch: master
    credentials: "{{ prefect.blocks.gitlab-credentials.mattijs-gitlab }}"

The reason I think it’s not being resolved (apart from not being able to see where in the code) is that I tried the following:

import prefect
from prefect.blocks.core import Block
from prefect.runner.storage import GitRepository

async with prefect.get_client() as client:
    cred = await Block.load("gitlab-credentials/mattijs-gitlab", client=client)

# This doesn't work
git_repo = GitRepository(
    url="<https://git_url/some_group/repo.git>",
    credentials=cred,
    branch="master",
    include_submodules=False,
)

# This does work
git_repo = GitRepository(
    url="<https://git_url/some_group/repo.git>",
    credentials={"token": MY_TOKEN},
    branch="master",
    include_submodules=False,
)

async with prefect.get_client() as client:
    await git_repo.pull_code()

do you happen to have the trace from when it failed? hmm also is there a reason you're passing the client like that?

async with prefect.get_client() as client:
    cred = await Block.load("gitlab-credentials/mattijs-gitlab", client=client)

shouldnt affect anything, just curious

Yes, relatively uninformative though (changed names of repo for confidentiality):

RuntimeError                              Traceback (most recent call last)
/home/jupyter-mattijs/repos/X/Y.ipynb Cell 24 line 2
      1 async with prefect.get_client() as client:
----> 2     await gr.pull_code()

File ~/.conda/envs/paps/lib/python3.10/site-packages/prefect/runner/storage.py:238, in GitRepository.pull_code(self)
    235 except subprocess.CalledProcessError as exc:
    236     # Hide the command used to avoid leaking the access token
    237     exc_chain = None if self._credentials else exc
--> 238     raise RuntimeError(
    239         f"Failed to clone repository {self._url!r} with exit code"
    240         f" {exc.returncode}."
    241     ) from exc_chain

RuntimeError: Failed to clone repository '<https://git_url/some_group/repo.git>' with exit code 128.

When you print the

cmd

at this line, it prints with the asterisks (*) for the token i.e.

https://********@git_url/some_group/repo.git

. The only thing that remains after from what I can see is the

anyio.run_process

function which I don’t think would call

.get_secret_value()

Oh just realised, I don’t have to include the client in the load function call

yep it should be the same thing, we will grab a client for you here with this `inject_client` decorator so you shouldn't have to pass your own unless you need a specific instance of a client for some reason

str

ohh, okay. i think i understand whats happening does this work?

git_repo = GitRepository(
    url="<https://git_url/some_group/repo.git>",
    credentials={"access_token": Block.load("gitlab-credentials/mattijs-gitlab").dict().get("token")},
    branch="master",
    include_submodules=False,
)

No, but if I understand what you’re trying to do then this does work:

git_repo = GitRepository(
    url="<https://git.ridewithvia.dev/data-science/prebooking-algo-params-snowflake.git>",
    credentials={"access_token": prefect.blocks.system.Secret(value=block.dict().get("token"))},
    branch="master",
    include_submodules=False,
)

(

Block.load()

straight didn’t work because that returns a coroutine, or am I doing something wrong?)

ah it depends whether the place you call it from has a running event loop bc that

@sync_compatible

decorator, so you'd have to await

Block.load

if calling from async context

@Nate so do you agree it’s a bug?

the conditional sync/async nature of Block.load is an explicit feature of

@sync_compatible

based on the type hinting I think we just need to better document this. GitRepository needs a

Secret

block or a string

git_repo = GitRepository(
    url="<https://git_url/some_group/repo.git>",
    credentials={"access_token": await Secret.load("my-gitlab-token")},
    branch="master",
    include_submodules=False,
)

an

enhancement

issue asking for support with

GitlabCredentials

here i think would also make sense

That’s not really the issue though. The issue is that you can’t use a

GitlabCredentials

block in a

prefect.yaml

pull step. Nor a

GithubCredentials

block I think but can’t test that. I think this is a regression.

hmm i see this now so it seems like we changed

git_clone

step under the hood to use

GitRepository

, which doesn't correctly template out Gitlab / Github repo blocks. so actually yeah I agree this is a bug

You should be able to use a

GitLabCredentials

block in this context, so looks like a bug to me also.

Okay, I don’t know what the recommended steps are? Shall I create bug report on github? I think maybe all that is required is to add the following here:

if isinstance(credentials.get("token"), SecretStr):
    credentials["token"] = credentials["token"].get_secret_value()

Yeah, if you create an issue someone should be able to pick it up and fix it before our next release.