Khoa Phan
11/20/2025, 10:45 AM- prefect.deployments.steps.git_clone:
id: clone-gitlab
branch: "main"
repository: "{{ prefect.blocks.secret.gitlaburl }}"
?Marvin
11/20/2025, 10:45 AMMarvin
11/20/2025, 10:47 AMbranch is the right key.
- id is optional and valid (lets you reference {{ clone-gitlab.directory }} later).
- {{ prefect.blocks.secret.gitlaburl }} is valid Jinja for a Secret block; you do not need .get().
A couple of caveats:
- Make sure the secret gitlaburl contains the full repository URL (e.g., <https://gitlab.com/org/repo.git> or <mailto:git@gitlab.com|git@gitlab.com>:org/repo.git).
- If the repository is private, you’ll also need to provide credentials. You can do this with either a GitLab credentials block or an access token secret.
Examples if it’s private:
- prefect.deployments.steps.git_clone:
id: clone-gitlab
repository: "<https://gitlab.com/org/repo.git>"
branch: "main"
credentials: "{{ prefect.blocks.gitlab-credentials.my-gitlab-creds }}"
or
- prefect.deployments.steps.git_clone:
id: clone-gitlab
repository: "<https://gitlab.com/org/repo.git>"
branch: "main"
access_token: "{{ prefect.blocks.secret.gitlab_token }}"
Note: don’t set both credentials and access_token at the same time.
If you share what your gitlaburl secret contains and whether the repo is private, I can confirm the exact configuration.Khoa Phan
11/20/2025, 10:52 AMgitlaburl. But keep getting: subprocess.CalledProcessError: Command [...--branch', 'main', '--depth', '1', '/app/prefect-cloud-3-main']' returned non-zero exit status 128.
helpMarvin
11/20/2025, 10:52 AMMarvin
11/20/2025, 10:53 AMgit clone almost always means “authentication failed”, “branch not found”, or “cannot reach host”. Given your secret value and the path shown (/app/prefect-cloud-3-main), here’s how to troubleshoot quickly.
Likely issues to check first:
1) Secret format (no angle brackets)
- Your message shows <https://oauth2>:<TOKEN>@.... The Secret value must be exactly:
<https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git>
No < > and no quotes. If you copied from Slack, it may have inserted angle brackets. Re‑save the Secret with the exact string above.
2) Token type and scopes
- For a GitLab Personal Access Token over HTTPS, the username must be oauth2 and the password is the token. Ensure the token has at least the read_api or read_repository scope for your GitLab version and access level.
- If you’re using a Deploy Token, the format is:
<https://USERNAME:TOKEN@gitlab.group.team/company/prefect-cloud-3.git>
(do NOT use oauth2 for deploy tokens).
3) Branch name
- The clone command is using --branch main. Confirm the repo actually has a main branch. If it’s master (or something else), either set the correct branch or remove the branch key to use the default.
4) URL-encoding
- If your token contains special characters (e.g., @, :, /, %), URL-encode it in the URL or switch to the access_token parameter approach below.
5) Network reachability from the worker
- If this runs in a container/remote worker, ensure it can resolve and reach gitlab.group.team. Exit 128 can also be “Could not resolve host” or TLS issues.
Recommended, simpler configuration
Instead of embedding the token in the repository URL, store only the token in a Secret and use the access_token field. This avoids URL formatting pitfalls:
- prefect.deployments.steps.git_clone:
id: clone-gitlab
repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
branch: "main"
access_token: "{{ prefect.blocks.secret.gitlab_token }}"
Or use a GitLab credentials block:
- prefect.deployments.steps.git_clone:
id: clone-gitlab
repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
branch: "main"
credentials: "{{ prefect.blocks.gitlab-credentials.my-gitlab-creds }}"
Quick test you can run to isolate the issue
- From the same environment as your worker:
git ls-remote <https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git>
If this fails, it will print a specific error like “Authentication failed”, “Repository not found”, or “Could not resolve host”.
Next info that would help me pinpoint it
- The exact stderr from the failed step (lines after “fatal:” or “error:”).
- Whether your default branch is main or something else.
- Whether the token is a PAT or a Deploy Token and which scopes it has.
- Confirmation that the Secret value has no < or > (and no quotes).Khoa Phan
11/20/2025, 10:57 AMMarvin
11/20/2025, 10:57 AMMarvin
11/20/2025, 11:00 AMgit ls-remote works on your machine.
Most likely culprits based on how the step runs:
1) Wrong branch name
- The step runs:
git clone --branch main --depth 1 <repo> /app/prefect-cloud-3-main
- If main doesn’t exist, clone returns 128 with “Remote branch main not found”.
- Check from the same runtime as your worker:
git ls-remote --heads <https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git> main
If that prints nothing, the branch isn’t there. Either remove branch: "main" or set it to the actual default branch.
2) Destination directory collision
- The step clones into /app/prefect-cloud-3-main. If that directory already exists and is not an empty git repo, git clone will fail with exit 128 (“destination path exists and is not an empty directory”).
- Exec into the worker (or add a quick debug step) and check:
ls -la /app
ls -la /app/prefect-cloud-3-main
If it’s there, remove it and re-run:
rm -rf /app/prefect-cloud-3-main
3) Environment parity vs. local
- Your local ls-remote doesn’t guarantee the worker can reach/auth to your GitLab (internal DNS, TLS trust, proxies).
- From the worker/container, run:
git ls-remote <https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git>
If this fails with SSL/cert or “could not resolve host”, fix network/CA trust in the worker.
4) Token formatting/scopes
- For GitLab PAT over HTTPS, the correct format is <https://oauth2:TOKEN@host/>....
- Ensure the token has read_repository (or api) scope.
- If you used a Deploy Token, it should be <https://USERNAME:TOKEN@host/>... (not oauth2).
Safer config to avoid URL formatting pitfalls
- Prefer passing the token separately:
- prefect.deployments.steps.git_clone:
id: clone-gitlab
repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
branch: "main"
access_token: "{{ prefect.blocks.secret.gitlab_token }}"
or with a GitLab credentials block:
- prefect.deployments.steps.git_clone:
id: clone-gitlab
repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
branch: "main"
credentials: "{{ prefect.blocks.gitlab-credentials.my-gitlab-creds }}"
Two quick discriminators
- Verify the branch exists:
git ls-remote --heads <https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git> main
- Check the target dir isn’t already there:
test -d /app/prefect-cloud-3-main && echo "exists"
If you can share the exact stderr line after “fatal:” from the failed run and confirm where the worker is running (Docker/K8s/VM), I can pinpoint which of these is the cause and give a precise fix.Khoa Phan
11/20/2025, 11:12 AMjob_variables empty ? @MarvinMarvin
11/20/2025, 11:12 AMMarvin
11/20/2025, 11:15 AMwork_pool:
name: "li-production-ecs-work-pool"
work_queue_name: null
job_variables: {}
or omit job_variables entirely:
work_pool:
name: "li-production-ecs-work-pool"
work_queue_name: null
If you want to avoid workers/pools entirely, run your flows with serve():
- In code:
from prefect import flow
@flow
def my_flow():
...
if __name__ == "__main__":
my_flow.serve(name="local-deployment", interval=300)
serve() doesn’t use a work pool or job_variables; it runs the “deployment” in the process that calls serve().
If your goal is to avoid ECS-specific fields (image, cpu, memory, etc.), consider:
- Using a Process work pool (simple local worker) with no job_variables
- Or a Managed work pool in Prefect Cloud where job_variables are optional and minimal
Tell me your execution target (local machine, Docker, ECS, etc.) and I can suggest the minimal config.Khoa Phan
11/20/2025, 11:17 AMMarvin
11/20/2025, 11:17 AMMarvin
11/20/2025, 11:21 AMls-remote works locally.
Most likely fixes:
1) Use credentials in a way ECS can actually apply
- If you used a GitLabCredentials block, your ECS image must have the integration installed. Without prefect-gitlab, the block can’t format oauth2:TOKEN@… and the clone will run unauthenticated.
- Fix: bake the integration into your image:
RUN pip install --no-cache-dir prefect-gitlab
- Easiest no-integration option: provide inline credentials with username/password so GitLab PATs are formatted correctly (oauth2 username):
- prefect.deployments.steps.git_clone:
repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
branch: "main"
credentials:
username: "oauth2"
password: "{{ prefect.blocks.secret.gitlab_token }}"
This works without installing prefect-gitlab.
2) Network and TLS from ECS
- If your GitLab is private/internal, ensure your ECS task subnets/security groups and routing can reach gitlab.group.team. Local success doesn’t guarantee ECS reachability.
- If your GitLab uses a corporate CA, add it to the image trust store, e.g.:
COPY corp-ca.crt /usr/local/share/ca-certificates/
RUN update-ca-certificates
Otherwise you may see: “SSL certificate problem: unable to get local issuer certificate” → exit 128.
3) Verify branch and destination directory
- If main doesn’t exist on that repo, git clone --branch main fails with 128. Check from inside a similar container:
git ls-remote --heads <https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git> main
- If the destination dir already exists (unlikely in fresh ECS tasks), clone will fail. The step tries /app/prefect-cloud-3-main.
4) Ensure the image has git
- Most slim base images don’t. If git is missing, you’d usually see a different error, but it’s still worth confirming:
RUN apt-get update && apt-get install -y git ca-certificates && rm -rf /var/lib/apt/lists/*
Recommended path for ECS
- Prefer inline credentials to avoid extra packages:
- prefect.deployments.steps.git_clone:
repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
branch: "main"
credentials:
username: "oauth2"
password: "{{ prefect.blocks.secret.gitlab_token }}"
- Or, if you want to use the GitLabCredentials block, bake prefect-gitlab into your ECS image.
- If your GitLab is internal or uses a custom CA, ensure ECS networking and CA trust are set up in the image/VPC.
If you can share the exact fatal message from git (e.g., “Authentication failed”, “Remote branch not found”, “SSL certificate problem”, “Could not resolve host”), I can pinpoint which of the above is the blocker and tailor the fix.