Maria Horgen
09/18/2025, 2:42 PMMarvin
09/18/2025, 2:42 PMMarvin
09/18/2025, 2:45 PMprefect.deployments.steps.pull.git_clone
(not prefect.deployments.steps.git_clone
)
About Azure DevOps + PAT:
- Do NOT base64 encode the PAT. Use the raw PAT.
- You can authenticate either by:
1) Supplying access_token:
and a clean HTTPS repo URL, or
2) Embedding a username and PAT in the URL (username can be anything non-empty)
Here are two working patterns you can try in your prefect.yaml.
Option A — use access_token (recommended; keep URL clean):
pull:
- prefect.deployments.steps.pull.git_clone:
repository: "<https://dev.azure.com/<org>/<project>/_git/<repo>>"
branch: "my-branch"
access_token: "${AZDO_PAT}"
- Set AZDO_PAT
in your worker/work-pool environment. The PAT should have at least “Code: Read” scope.
Option B — embed username + PAT in the URL (ensure URL-encoding if needed):
pull:
- prefect.deployments.steps.pull.git_clone:
repository: "<https://pat:${AZDO_PAT}@dev.azure.com/<org>/<project>/_git/<repo>>"
branch: "my-branch"
- Any non-empty username is fine; pat
is a common placeholder.
- If your PAT contains special characters, URL-encode it before embedding.
Common causes of exit code 128:
- Auth failed: wrong/missing PAT, missing scope, or malformed URL
- PAT accidentally base64-encoded
- The branch name doesn’t exist (set branch:
correctly)
- The PAT is not available in the worker environment (e.g., env var not set)
- Typos in the Azure DevOps path (verify with a manual git ls-remote
on the worker)
Quick verification tips:
- On the same machine/environment your worker runs, try:
- Using Option B’s URL directly with git ls-remote <https://pat:<PAT>@dev.azure.com/<org>/<project>/_git/<repo>>
- Confirm the branch exists and the PAT has “Code (Read)” scope and org access
- If it still fails, capture the full stderr from the step (not just exit code) — that often points to the exact auth or URL parsing issue.
Docs:
- Prefect step reference: prefect.deployments.steps.pull.git_clone
- Prefect YAML guide: prefect.yaml how-to
- Azure DevOps PAT usage: Use personal access tokens - Azure DevOps
If you can share (sanitized) your current prefect.yaml
step and the exact error output, I can pinpoint the issue quickly.Maria Horgen
09/19/2025, 10:38 AM- prefect.deployments.steps.run_shell_script:
script: uv sync
directory: "{{ git-clone.directory }}"
This fails with the error: run_shell_script
failed with error code 2: error: No pyproject.toml
found in current directory or any parent directory, even though the pyproject.toml file is in the root of the cloned repo. How can I fix this?Marvin
09/19/2025, 10:38 AMMarvin
09/19/2025, 10:41 AM{{ git-clone.directory }}
won’t resolve and the command runs in the default working dir.
Try this:
- Give the clone step an id
- Either set the working directory once, or pass it to run_shell_script
Example 1 — set working directory once:
pull:
- prefect.deployments.steps.pull.git_clone:
id: git_clone
repository: "<https://dev.azure.com/<org>/<project>/_git/<repo>>"
branch: "my-branch"
- prefect.deployments.steps.pull.set_working_directory:
directory: "{{ git_clone.directory }}"
- prefect.deployments.steps.utility.run_shell_script:
script: uv sync
Example 2 — pass directory to the shell step:
pull:
- prefect.deployments.steps.pull.git_clone:
id: git_clone
repository: "<https://dev.azure.com/<org>/<project>/_git/<repo>>"
branch: "my-branch"
- prefect.deployments.steps.utility.run_shell_script:
script: uv sync
directory: "{{ git_clone.directory }}"
Notes:
- Use the canonical step path for the shell step: prefect.deployments.steps.utility.run_shell_script
- If your project lives in a subdirectory of the repo, point to it:
- directory: "{{ git_clone.directory }}/path/to/project"
- To debug quickly, add a step:
- prefect.deployments.steps.utility.run_shell_script:
script: "pwd && ls -la"
directory: "{{ git_clone.directory }}"
Docs:
- git_clone: prefect.deployments.steps.pull.git_clone
- set_working_directory: prefect.deployments.steps.pull.set_working_directory
- run_shell_script: prefect.deployments.steps.utility.run_shell_script
If it still fails, share your full pull section and I’ll spot-check the templating and paths.Maria Horgen
09/19/2025, 11:03 AM- prefect.deployments.steps.pull.set_working_directory:
directory: "{{ git_clone.directory }}/prefect"
This points to the subdirectory where my flow code are stored, and also the prefect.yaml file, assuming that the {{ git_clone.directory }}
is the root of my cloned repo. Inside the prefect-folder I have a new folder called flows, so that the entrypoint in my deployment is: flows/my_flow.py:my_flow
. However, the deployment is failing with the error:
[Errno 2] No such file or directory: '/prefect'
Indicating that the set-directory-step is failingMarvin
09/19/2025, 11:03 AMMarvin
09/19/2025, 11:05 AM{{ git_clone.directory }}
to an empty string, so your template became just /prefect
. That means the git_clone step’s output isn’t being found by the template.
Most common causes:
- The step id isn’t attached correctly to the git_clone step (indentation/YAML structure)
- The referenced id in the template doesn’t exactly match the step id
- set_working_directory runs before git_clone
- The step path is slightly off
Use this minimal working example as a template:
pull:
- prefect.deployments.steps.pull.git_clone:
id: git_clone
repository: "<https://dev.azure.com/<org>/<project>/_git/<repo>>"
branch: "my-branch"
# Optional: Debug what the template resolves to
- prefect.deployments.steps.utility.run_shell_script:
script: |
echo "Repo dir: {{ git_clone.directory }}"
ls -la "{{ git_clone.directory }}"
ls -la "{{ git_clone.directory }}/prefect"
- prefect.deployments.steps.pull.set_working_directory:
directory: "{{ git_clone.directory }}/prefect"
- prefect.deployments.steps.utility.run_shell_script:
script: uv sync
Checklist:
- Ensure the id is nested under the git_clone step (as shown). This is the correct syntax:
- prefect.deployments.steps.pull.git_clone:
id: git_clone
...
- The template must match the id exactly: {{ git_clone.directory }}
- set_working_directory must come after the git_clone step
- The canonical path for the shell step is prefect.deployments.steps.utility.run_shell_script
(not prefect.deployments.steps.run_shell_script
)
If the debug step prints an empty line for “Repo dir:”, the id reference isn’t resolving. In that case, please paste your full pull section and I’ll spot the mismatch.
Docs for reference:
- git_clone: prefect.deployments.steps.pull.git_clone
- set_working_directory: prefect.deployments.steps.pull.set_working_directory
- run_shell_script: prefect.deployments.steps.utility.run_shell_scriptMaria Horgen
09/19/2025, 11:10 AMrun_shell_script
allows for the execution of one or more shell commands in a subprocess, and returns the standard output and standard error of the script. This step is useful for scripts that require execution in a specific environment, or those which have specific input and output requirements. Note that setting stream_output: true
for run_shell_script
writes the output and error to stdout in the execution environment, which will not be sent to the Prefect API.Marvin
09/19/2025, 11:10 AMMarvin
09/19/2025, 11:11 AMpull:
- prefect.deployments.steps.pull.git_clone:
id: git_clone
repository: "<https://dev.azure.com/<org>/<project>/_git/<repo>>"
branch: "my-branch"
# Quick debug: verify the resolved directory on the worker
- prefect.deployments.steps.utility.run_shell_script:
script: |
echo "Repo dir: {{ git_clone.directory }}"
ls -la "{{ git_clone.directory }}"
ls -la "{{ git_clone.directory }}/prefect"
stream_output: true
# Option A: set CWD for subsequent steps
- prefect.deployments.steps.pull.set_working_directory:
directory: "{{ git_clone.directory }}/prefect"
# Option B (simpler/less stateful): pass directory directly to each script
- prefect.deployments.steps.utility.run_shell_script:
script: uv sync
directory: "{{ git_clone.directory }}/prefect"
stream_output: true
Checklist:
- The id must be nested under the git_clone step itself (indentation matters)
- The reference must match exactly: {{ git_clone.directory }}
- set_working_directory must come after git_clone
- Use the canonical paths:
- prefect.deployments.steps.pull.git_clone
- prefect.deployments.steps.pull.set_working_directory
- prefect.deployments.steps.utility.run_shell_script
If the debug step prints “Repo dir:” as empty or errors on the ls command, the template isn’t resolving. In that case, please paste your full pull section and I’ll spot any indentation/id/path issues.
Docs for reference:
- git_clone: prefect.deployments.steps.pull.git_clone
- set_working_directory: prefect.deployments.steps.pull.set_working_directory
- run_shell_script: prefect.deployments.steps.utility.run_shell_scriptMaria Horgen
09/19/2025, 11:14 AMFileNotFoundError: [Errno 2] No such file or directory: 'my_repo-my-branch/prefect'
even though I know that I have a folder named prefect in the root of my repoMarvin
09/19/2025, 11:14 AMMarvin
09/19/2025, 11:18 AMpull:
- prefect.deployments.steps.pull.set_working_directory:
directory: "/tmp"
- prefect.deployments.steps.pull.git_clone:
id: git_clone
repository: "<https://dev.azure.com/<org>/<project>/_git/<repo>>"
branch: "my-branch"
- prefect.deployments.steps.pull.set_working_directory:
directory: "{{ git_clone.directory }}/prefect"
- prefect.deployments.steps.utility.run_shell_script:
script: uv sync
Option B — don’t change global CWD; point run_shell_script at the repo folder
- Less stateful and easier to debug.
pull:
- prefect.deployments.steps.pull.set_working_directory:
directory: "/tmp"
- prefect.deployments.steps.pull.git_clone:
id: git_clone
repository: "<https://dev.azure.com/<org>/<project>/_git/<repo>>"
branch: "my-branch"
- prefect.deployments.steps.utility.run_shell_script:
script: uv sync
directory: "{{ git_clone.directory }}/prefect"
Tip: Add a quick sanity check step to confirm the path on the worker (logs show up in Managed Worker logs, not the flow run UI):
- prefect.deployments.steps.utility.run_shell_script:
script: |
echo "PWD: $(pwd)"
echo "Repo dir: {{ git_clone.directory }}"
ls -la "{{ git_clone.directory }}"
ls -la "{{ git_clone.directory }}/prefect"
stream_output: true
Relevant refs:
- set_working_directory: prefect.deployments.steps.pull.set_working_directory
- git_clone: prefect.deployments.steps.pull.git_clone
- run_shell_script: prefect.deployments.steps.utility.run_shell_script
If you prefer a different base than /tmp, just set that in the first set_working_directory step and everything else will line up.