Ridhwan
08/28/2025, 3:56 PMMarvin
08/28/2025, 3:56 PMRidhwan
08/28/2025, 4:02 PMRidhwan
08/28/2025, 4:02 PMMarvin
08/28/2025, 4:02 PMMarvin
08/28/2025, 4:02 PMgit_clone
step does not accept a destination path. It clones into a temporary directory and returns that path as directory
. In your prefect.yaml, you typically reference that output and set it as the working directory for subsequent steps.
Here’s the recommended pattern in Prefect 3.x:
deployments:
- name: my-deployment
# ...
pull:
- prefect.deployments.steps.git_clone:
id: clone
repository: "<https://github.com/owner/repo.git>"
branch: "main"
# optional: only pull specific folders (sparse checkout)
# directories: ["flows", "libs/common"]
- prefect.deployments.steps.set_working_directory:
directory: "{{ clone.directory }}"
Notes:
- {{ clone.directory }}
references the output from the previous step with id: clone
.
- The directories
parameter performs a sparse checkout (only those subfolders), not a destination path override.
If you truly need to clone into a fixed path like C/user/user/mydir
- Either bypass git_clone
and run Git yourself:
deployments:
- name: my-deployment
pull:
- prefect.deployments.steps.set_working_directory:
directory: "C:/user/user/mydir" # or "C:\\user\\user\\mydir"
- prefect.deployments.steps.utility.run_shell_script:
commands:
- 'git clone --branch main <https://github.com/owner/repo.git> .'
- Or clone with git_clone
then move/copy files into your fixed directory using utility.run_shell_script
.
Windows path tips:
- Prefer forward slashes in YAML: C:/user/user/mydir
- Or escape backslashes: C:\\user\\user\\mydir
Docs:
- API: pull steps (git_clone, set_working_directory)
- Guide: prefect.yaml and step templatingMarvin
08/28/2025, 4:03 PMprefect.deployments.steps.git_clone
step does not let you choose a destination folder. It clones into a managed working directory. If you need a specific path (like C:/user/user/mydir
), use a shell step to clone/pull to that path, then set the working directory.
Two common patterns:
1) Clone and run from a subdirectory inside the repo (most common)
pull:
- prefect.deployments.steps.git_clone:
repository: <https://github.com/owner/repo.git>
branch: main
# optional: only bring over specific subfolders; NOT a destination path
# directories:
# - flows
- prefect.deployments.steps.set_working_directory:
# relative to the cloned repo root
directory: flows/my_flow
2) Clone/pull into an explicit absolute path (e.g., Windows), then set it as the working dir
pull:
- prefect.deployments.steps.run_shell_script:
script: |
REM Pull if the repo already exists, otherwise clone
if exist "C:\user\user\mydir\.git" (
git -C "C:\user\user\mydir" pull
) else (
git clone <https://github.com/owner/repo.git> "C:\user\user\mydir"
)
- prefect.deployments.steps.set_working_directory:
directory: C:\user\user\mydir
Notes:
- The directories
key on git_clone
is for selecting subdirectories from the repo to include; it is not a destination path.
- If your worker runs in Docker or Kubernetes, an absolute Windows path won’t exist inside the container—use relative paths within the container’s filesystem instead.
Docs for reference:
- prefect.yaml guide
- git_clone and set_working_directory API
If you can share whether your worker is running on Windows directly vs. in Docker/Kubernetes, I can tailor the exact config.Ridhwan
08/28/2025, 4:07 PMMarvin
08/28/2025, 4:07 PMMarvin
08/28/2025, 4:10 PMprefect.deployments.steps.git_clone
in Prefect 3.x, each run clones into a fresh, managed directory. It does not overwrite a previous clone and it does not reuse the same path across runs. The step returns the clone path as directory
, which you can reference like {{ clone.directory }}
in subsequent steps.
If you want to reuse a fixed location (and avoid recloning), you’ll need to manage it yourself with a shell step, e.g., detect the folder and run git pull
if it exists or git clone
if it doesn’t. In that pattern, repeated runs will reuse that directory, and your script dictates whether it updates or overwrites.
References:
- git_clone and set_working_directory API
- prefect.yaml guide