<@ULVA73B9P> is the following block correct: ```- ...
# ask-marvin
k
@Marvin is the following block correct:
Copy code
- prefect.deployments.steps.git_clone:
      id: clone-gitlab
      branch: "main"
      repository: "{{ prefect.blocks.secret.gitlaburl }}"
?
m
thought for 146.5 seconds
Short answer: Yes — that step is syntactically correct for Prefect 3.x. -
branch
is the right key. -
id
is optional and valid (lets you reference
{{ clone-gitlab.directory }}
later). -
{{ prefect.blocks.secret.gitlaburl }}
is valid Jinja for a Secret block; you do not need
.get()
. A couple of caveats: - Make sure the secret
gitlaburl
contains the full repository URL (e.g.,
<https://gitlab.com/org/repo.git>
or
<mailto:git@gitlab.com|git@gitlab.com>:org/repo.git
). - If the repository is private, you’ll also need to provide credentials. You can do this with either a GitLab credentials block or an access token secret. Examples if it’s private:
Copy code
- prefect.deployments.steps.git_clone:
    id: clone-gitlab
    repository: "<https://gitlab.com/org/repo.git>"
    branch: "main"
    credentials: "{{ prefect.blocks.gitlab-credentials.my-gitlab-creds }}"
or
Copy code
- prefect.deployments.steps.git_clone:
    id: clone-gitlab
    repository: "<https://gitlab.com/org/repo.git>"
    branch: "main"
    access_token: "{{ prefect.blocks.secret.gitlab_token }}"
Note: don’t set both
credentials
and
access_token
at the same time. If you share what your
gitlaburl
secret contains and whether the repo is private, I can confirm the exact configuration.
k
@Marvin https://oauth2:<TOKEN>@gitlab.group.team/company/prefect-cloud-3.git that is my secret
gitlaburl
. But keep getting: subprocess.CalledProcessError: Command [...--branch', 'main', '--depth', '1', '/app/prefect-cloud-3-main']' returned non-zero exit status 128. help
m
thought for 27.3 seconds
Thanks for the details — exit status 128 from
git clone
almost always means “authentication failed”, “branch not found”, or “cannot reach host”. Given your secret value and the path shown (
/app/prefect-cloud-3-main
), here’s how to troubleshoot quickly. Likely issues to check first: 1) Secret format (no angle brackets) - Your message shows
<https://oauth2>:<TOKEN>@...
. The Secret value must be exactly:
Copy code
<https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git>
No
<
>
and no quotes. If you copied from Slack, it may have inserted angle brackets. Re‑save the Secret with the exact string above. 2) Token type and scopes - For a GitLab Personal Access Token over HTTPS, the username must be
oauth2
and the password is the token. Ensure the token has at least the
read_api
or
read_repository
scope for your GitLab version and access level. - If you’re using a Deploy Token, the format is:
Copy code
<https://USERNAME:TOKEN@gitlab.group.team/company/prefect-cloud-3.git>
(do NOT use
oauth2
for deploy tokens). 3) Branch name - The clone command is using
--branch main
. Confirm the repo actually has a
main
branch. If it’s
master
(or something else), either set the correct branch or remove the
branch
key to use the default. 4) URL-encoding - If your token contains special characters (e.g.,
@
,
:
,
/
,
%
), URL-encode it in the URL or switch to the
access_token
parameter approach below. 5) Network reachability from the worker - If this runs in a container/remote worker, ensure it can resolve and reach
gitlab.group.team
. Exit 128 can also be “Could not resolve host” or TLS issues. Recommended, simpler configuration Instead of embedding the token in the repository URL, store only the token in a Secret and use the
access_token
field. This avoids URL formatting pitfalls:
Copy code
- prefect.deployments.steps.git_clone:
    id: clone-gitlab
    repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
    branch: "main"
    access_token: "{{ prefect.blocks.secret.gitlab_token }}"
Or use a GitLab credentials block:
Copy code
- prefect.deployments.steps.git_clone:
    id: clone-gitlab
    repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
    branch: "main"
    credentials: "{{ prefect.blocks.gitlab-credentials.my-gitlab-creds }}"
Quick test you can run to isolate the issue - From the same environment as your worker:
Copy code
git ls-remote <https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git>
If this fails, it will print a specific error like “Authentication failed”, “Repository not found”, or “Could not resolve host”. Next info that would help me pinpoint it - The exact stderr from the failed step (lines after “fatal:” or “error:”). - Whether your default branch is
main
or something else. - Whether the token is a PAT or a Deploy Token and which scopes it has. - Confirmation that the Secret value has no
<
or
>
(and no quotes).
k
@Marvin git ls-remote https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git this works for me very well. I tried with only the credentials block but same error. What is the cause of it ?
m
thought for 153.0 seconds
Thanks — a few things can cause exit 128 in this step even if
git ls-remote
works on your machine. Most likely culprits based on how the step runs: 1) Wrong branch name - The step runs:
Copy code
git clone --branch main --depth 1 <repo> /app/prefect-cloud-3-main
- If
main
doesn’t exist, clone returns 128 with “Remote branch main not found”. - Check from the same runtime as your worker:
Copy code
git ls-remote --heads <https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git> main
If that prints nothing, the branch isn’t there. Either remove
branch: "main"
or set it to the actual default branch. 2) Destination directory collision - The step clones into
/app/prefect-cloud-3-main
. If that directory already exists and is not an empty git repo,
git clone
will fail with exit 128 (“destination path exists and is not an empty directory”). - Exec into the worker (or add a quick debug step) and check:
Copy code
ls -la /app
  ls -la /app/prefect-cloud-3-main
If it’s there, remove it and re-run:
Copy code
rm -rf /app/prefect-cloud-3-main
3) Environment parity vs. local - Your local
ls-remote
doesn’t guarantee the worker can reach/auth to your GitLab (internal DNS, TLS trust, proxies). - From the worker/container, run:
Copy code
git ls-remote <https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git>
If this fails with SSL/cert or “could not resolve host”, fix network/CA trust in the worker. 4) Token formatting/scopes - For GitLab PAT over HTTPS, the correct format is
<https://oauth2:TOKEN@host/>...
. - Ensure the token has
read_repository
(or
api
) scope. - If you used a Deploy Token, it should be
<https://USERNAME:TOKEN@host/>...
(not
oauth2
). Safer config to avoid URL formatting pitfalls - Prefer passing the token separately:
Copy code
- prefect.deployments.steps.git_clone:
      id: clone-gitlab
      repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
      branch: "main"
      access_token: "{{ prefect.blocks.secret.gitlab_token }}"
or with a GitLab credentials block:
Copy code
- prefect.deployments.steps.git_clone:
      id: clone-gitlab
      repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
      branch: "main"
      credentials: "{{ prefect.blocks.gitlab-credentials.my-gitlab-creds }}"
Two quick discriminators - Verify the branch exists:
Copy code
git ls-remote --heads <https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git> main
- Check the target dir isn’t already there:
Copy code
test -d /app/prefect-cloud-3-main && echo "exists"
If you can share the exact stderr line after “fatal:” from the failed run and confirm where the worker is running (Docker/K8s/VM), I can pinpoint which of these is the cause and give a precise fix.
k
variables: &shared_variables image: "481150380906.dkr.ecr.eu-west-1.amazonaws.com/prefect-cloud:latest" execution_role_arn: "{{ prefect.variables.execution_role_arn }}" task_role_arn: "{{ prefect.variables.task_role_arn }}" cluster: "{{ prefect.variables.cluster }}" configure_cloudwatch_logs: true launch_type: "FARGATE" cpu: 2048 memory: 8192 env: PREFECT_RESULTS_PERSIST_BY_DEFAULT: "true" PREFECT_TASKS_DEFAULT_PERSIST_RESULT: "true" work_pool_config: &work_pool_config work_pool: name: "li-production-ecs-work-pool" work_queue_name: null job_variables: <<: *shared_variables I don't want to use work_pool. Can I make
job_variables
empty ? @Marvin
m
thought for 164.4 seconds
Short answer: - For deployments created via prefect.yaml or flow.deploy(), a work pool is required in Prefect 3.x. - You can set job_variables to empty ({}), or just omit it. But you still need to specify a work pool name. - If you truly don’t want to use a work pool, use serve() instead of deploy(). Concretely: - Empty job_variables is valid:
Copy code
work_pool:
  name: "li-production-ecs-work-pool"
  work_queue_name: null
  job_variables: {}
or omit job_variables entirely:
Copy code
work_pool:
  name: "li-production-ecs-work-pool"
  work_queue_name: null
If you want to avoid workers/pools entirely, run your flows with serve(): - In code:
Copy code
from prefect import flow

@flow
def my_flow():
    ...

if __name__ == "__main__":
    my_flow.serve(name="local-deployment", interval=300)
serve() doesn’t use a work pool or job_variables; it runs the “deployment” in the process that calls serve(). If your goal is to avoid ECS-specific fields (image, cpu, memory, etc.), consider: - Using a Process work pool (simple local worker) with no job_variables - Or a Managed work pool in Prefect Cloud where job_variables are optional and minimal Tell me your execution target (local machine, Docker, ECS, etc.) and I can suggest the minimal config.
k
@Marvin I use ECS workpool hence the issue with Gitlab. Is there something I have to change ?
m
thought for 220.1 seconds
Got it — since you’re running in an ECS work pool, the git_clone step executes inside your ECS task container. There are a few ECS-specific gotchas that commonly cause git exit 128 even when
ls-remote
works locally. Most likely fixes: 1) Use credentials in a way ECS can actually apply - If you used a GitLabCredentials block, your ECS image must have the integration installed. Without
prefect-gitlab
, the block can’t format
oauth2:TOKEN@…
and the clone will run unauthenticated. - Fix: bake the integration into your image:
Copy code
RUN pip install --no-cache-dir prefect-gitlab
- Easiest no-integration option: provide inline credentials with username/password so GitLab PATs are formatted correctly (oauth2 username):
Copy code
- prefect.deployments.steps.git_clone:
      repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
      branch: "main"
      credentials:
        username: "oauth2"
        password: "{{ prefect.blocks.secret.gitlab_token }}"
This works without installing
prefect-gitlab
. 2) Network and TLS from ECS - If your GitLab is private/internal, ensure your ECS task subnets/security groups and routing can reach
gitlab.group.team
. Local success doesn’t guarantee ECS reachability. - If your GitLab uses a corporate CA, add it to the image trust store, e.g.:
Copy code
COPY corp-ca.crt /usr/local/share/ca-certificates/
  RUN update-ca-certificates
Otherwise you may see: “SSL certificate problem: unable to get local issuer certificate” → exit 128. 3) Verify branch and destination directory - If
main
doesn’t exist on that repo,
git clone --branch main
fails with 128. Check from inside a similar container:
Copy code
git ls-remote --heads <https://oauth2:TOKEN@gitlab.group.team/company/prefect-cloud-3.git> main
- If the destination dir already exists (unlikely in fresh ECS tasks), clone will fail. The step tries
/app/prefect-cloud-3-main
. 4) Ensure the image has git - Most slim base images don’t. If
git
is missing, you’d usually see a different error, but it’s still worth confirming:
Copy code
RUN apt-get update && apt-get install -y git ca-certificates && rm -rf /var/lib/apt/lists/*
Recommended path for ECS - Prefer inline credentials to avoid extra packages:
Copy code
- prefect.deployments.steps.git_clone:
      repository: "<https://gitlab.group.team/company/prefect-cloud-3.git>"
      branch: "main"
      credentials:
        username: "oauth2"
        password: "{{ prefect.blocks.secret.gitlab_token }}"
- Or, if you want to use the GitLabCredentials block, bake
prefect-gitlab
into your ECS image. - If your GitLab is internal or uses a custom CA, ensure ECS networking and CA trust are set up in the image/VPC. If you can share the exact fatal message from git (e.g., “Authentication failed”, “Remote branch not found”, “SSL certificate problem”, “Could not resolve host”), I can pinpoint which of the above is the blocker and tailor the fix.