<@ULVA73B9P> I have a deployment that uses github ...
# ask-marvin
f
@Marvin I have a deployment that uses github authentication import os from prefect import flow from prefect.runner.storage import GitRepository from prefect_github import GitHubCredentials github_credentials_block = GitHubCredentials.load("my-github-credentials-block") @flow(_log_prints_=True) def buy(): print("Starting template flow...") print("Buyin g securities") print("Template flow completed successfully") return
Copy code
{
  "status": "success",
  "message": "Template flow executed successfully"
}
if name == "__main__": # Get GitHub repository URL and commit SHA from environment or use defaults github_repo = os.getenv("GITHUB_REPOSITORY_URL", "https://github.com/repo") github_sha = os.getenv("GITHUB_SHA", "main") # Deploy using Git storage buy.from_source( source=GitRepository( url=github_repo, _commit_sha_=github_sha, credentials=github_credentials_block ), entrypoint="datasets/flows/template/deploy.py:buy" ).deploy( name="template-deployment", _work_pool_name_="dockerhub-pool-test", _job_variables_={ "image_pull_policy": "Always", "env": { "PREFECT_LOGGING_LEVEL": "DEBUG", "PYTHONUNBUFFERED": "1" }, } ) however, when running the worflow the authentication fails during the cloning of the repo Command '['git', 'clone', 'repo', '--filter=blob:none', '--no-checkout', '/app/data_workflow_prefect']' returned non-zero exit status 128.
m
thought for 349.7 seconds
Thanks for sharing the snippet — a few things jump out that commonly cause “git clone … exited with status 128” when using Git storage with authentication. Quick checks first - Make sure the repository URL is a valid HTTPS URL (and not Slack’s angle-bracket formatting). In your code I see a default of
"<https://github.com/repo>"
. The literal angle brackets will break git. Use
<https://github.com/<owner>/<repo>.git>
with no brackets and include the
.git
suffix. - Only set
commit_sha
when you actually have a SHA. In your example,
GITHUB_SHA
falls back to
"main"
, but
commit_sha="main"
is not a SHA. Use
branch="main"
for a branch, and reserve
commit_sha
for a 40‑char commit id. - Confirm the token has read access to the repo (PAT needs repo scope; GitHub App installation tokens must be installed on that repo). - Turn on git debug to see the precise auth error: - Add to your deployment’s `job_variables.env`:
GIT_TRACE=1
and
GIT_CURL_VERBOSE=1
Two supported ways to provide credentials A) Keep using the GitHubCredentials block (prefect-github) - Ensure the runtime environment (the worker image) has the prefect-github package installed. If the worker can’t deserialize the
GitHubCredentials
block type at runtime, Prefect will attempt to clone without creds and you’ll see exit 128. - Install at both build and run time: - If you’re using Docker workers, include
pip install -U prefect-github
in your image or use a requirements file. - Make sure you already ran
prefect block register -m prefect_github
when you created the block. - Example with fixes applied:
Copy code
import os
from prefect import flow
from prefect.runner.storage import GitRepository
from prefect_github import GitHubCredentials

github_credentials = GitHubCredentials.load("my-github-credentials-block")

@flow(log_prints=True)
def buy():
    print("Starting template flow...")
    print("Buying securities")
    print("Template flow completed successfully")
    return {"status": "success", "message": "Template flow executed successfully"}

if __name__ == "__main__":
    repo_url = os.getenv("GITHUB_REPOSITORY_URL", "<https://github.com/owner/repo.git>")
    branch = os.getenv("GIT_BRANCH", "main")
    commit = os.getenv("GITHUB_SHA")  # only use if it's a real SHA

    storage = GitRepository(
        url=repo_url,
        branch=None if commit else branch,
        commit_sha=commit,  # set only if commit is a real SHA
        credentials=github_credentials,
    )

    buy.from_source(
        source=storage,
        entrypoint="datasets/flows/template/deploy.py:buy",
    ).deploy(
        name="template-deployment",
        work_pool_name="dockerhub-pool-test",
        job_variables={
            "image_pull_policy": "Always",
            "env": {
                "PREFECT_LOGGING_LEVEL": "DEBUG",
                "PYTHONUNBUFFERED": "1",
                "GIT_TRACE": "1",
                "GIT_CURL_VERBOSE": "1",
            },
        },
    )
B) Use a token directly via GitRepository credentials - This avoids needing prefect-github in the worker image. Store your token in a Prefect Secret block, then pass it as an access_token. ``` import os from prefect import flow from prefect.runner.storage import GitRepository from prefect.blocks.system import Secret @flow(log_prints=True) def buy(): print("Starting template flow...") print("Buying securities") print("Template flow completed successfully") return {"status": "success", "message": "Template flow executed successfully"} if name == "__main__": repo_url = os.getenv("GITHUB_REPOSITORY_URL", "https://github.com/owner/repo.git") branch = os.getenv("GIT_BRANCH", "main") commit = os.getenv("GITHUB_SHA") # only if a real SHA token = Secret.load("github-access-token").get() storage = GitRepository( url=repo_url, branch=None if commit else branch, commit_sha=commit, credentials={"access_token": token}, )
buy.from_source( source=storage, entrypoint="datasets/flows/template/deploy.py:buy", ).deploy( name="template-deployment", work_pool_name="dockerhub-pool-test", job_variables={ "image_pull_policy": "Always", "env": { "PREFECT_LOGGING_LEVEL": "DEBUG", "PYTHONUNBUFFERED": "1", "GIT_TRACE": "1", "GIT_CURL_VERBOSE": "1", }, }, ) ``` Common reasons for git exit status 128 in this setup - URL is malformed (angle brackets, missing .git, wrong owner/repo). - Credentials aren’t present at runtime: - Using a GitHubCredentials block but the worker image does not have prefect-github installed, so the block can’t be resolved. - Token missing/empty, wrong secret name, or not passed to
GitRepository
. - Token lacks required scope or was revoked. - Using SSH URL without SSH keys/known_hosts in the container. - Passing a branch name in
commit_sha
instead of using the
branch
field. A couple of quick questions to pinpoint your case - What is the exact value of
GITHUB_REPOSITORY_URL
at runtime? If the logs show
git clone repo
, it suggests the URL may just be the literal string
repo
. - Which worker type/image are you using for
dockerhub-pool-test
? Does it include
prefect-github
if you’re using the GitHubCredentials block? - Is
GITHUB_SHA
a real SHA (from CI) or the string
main
? If the latter, switch to
branch="main"
instead of
commit_sha
. Helpful docs - Git storage for deployments: https://docs-3.prefect.io/v3/how-to-guides/deployments/store-flow-code - GitRepository API: https://docs-3.prefect.io/v3/api-ref/python/prefect-runner-storage/#prefect.runner.storage.GitRepository - prefect-github integration: https://docs-3.prefect.io/integrations/prefect-github/ If you can share a snippet of the worker logs around the git clone (with
GIT_TRACE=1
), the resolved repo URL, and your worker image, I can help zero in on the exact fix.
f
Copy code
@Marvin 
"""
Deploy script for template flow
"""
import os
from prefect import flow
from prefect.runner.storage import GitRepository
from prefect_github import GitHubCredentials
github_credentials_block = GitHubCredentials.load("my-github-credentials-block")

@flow(log_prints=True)
def buy():
    print("Starting template flow...")
    print("Buying securities")
    print("Template flow completed successfully")
    return {"status": "success", "message": "Template flow executed successfully"}


if __name__ == "__main__":
    # Get GitHub repository URL and branch/commit from environment or use defaults
    github_repo = os.getenv("GITHUB_REPOSITORY_URL", "<https://github.com/repo.git>")
    branch = os.getenv("GIT_BRANCH", "main")
    commit = os.getenv("GITHUB_SHA")  # only use if it's a real SHA
    
    # Create Git storage with proper branch/commit handling
    storage = GitRepository(
        url=github_repo,
        branch=None if commit else branch,
        commit_sha=commit, 
        credentials=github_credentials_block,
    )

    # Deploy using Git storage
    buy.from_source(
        source=storage,
        entrypoint="datasets/flows/template/deploy.py:buy"
    ).deploy(
        name="template-deployment",
        work_pool_name="dockerhub-pool-test",
        job_variables={
            "image_pull_policy": "Always",
            "env": {
                "PREFECT_LOGGING_LEVEL": "DEBUG",
                "PYTHONUNBUFFERED": "1",
                "GIT_TRACE": "1",
                "GIT_CURL_VERBOSE": "1",
            },
        }
    )