<@ULVA73B9P> tell me how to specify the directory ...
# ask-marvin
r
@Marvin tell me how to specify the directory to pull in the yaml config here. for example, i want to git pull in the wkdir <c://user/user/mydir/> prefect.deployments.steps.git_clone
m
thought for 361.1 seconds
r
@\
@Marvin tell me how to specify the directory to pull in the yaml config here. for example, i want to git pull in the wkdir <c://user/user/mydir/> prefect.deployments.steps.git_clone
m
thought for 65.4 seconds
Short answer: the
git_clone
step does not accept a destination path. It clones into a temporary directory and returns that path as
directory
. In your prefect.yaml, you typically reference that output and set it as the working directory for subsequent steps. Here’s the recommended pattern in Prefect 3.x:
Copy code
deployments:
  - name: my-deployment
    # ...
    pull:
      - prefect.deployments.steps.git_clone:
          id: clone
          repository: "<https://github.com/owner/repo.git>"
          branch: "main"
          # optional: only pull specific folders (sparse checkout)
          # directories: ["flows", "libs/common"]
      - prefect.deployments.steps.set_working_directory:
          directory: "{{ clone.directory }}"
Notes: -
{{ clone.directory }}
references the output from the previous step with
id: clone
. - The
directories
parameter performs a sparse checkout (only those subfolders), not a destination path override. If you truly need to clone into a fixed path like C/user/user/mydir - Either bypass
git_clone
and run Git yourself:
Copy code
deployments:
  - name: my-deployment
    pull:
      - prefect.deployments.steps.set_working_directory:
          directory: "C:/user/user/mydir"   # or "C:\\user\\user\\mydir"
      - prefect.deployments.steps.utility.run_shell_script:
          commands:
            - 'git clone --branch main <https://github.com/owner/repo.git> .'
- Or clone with
git_clone
then move/copy files into your fixed directory using
utility.run_shell_script
. Windows path tips: - Prefer forward slashes in YAML:
C:/user/user/mydir
- Or escape backslashes:
C:\\user\\user\\mydir
Docs: - API: pull steps (git_clone, set_working_directory) - Guide: prefect.yaml and step templating
Short answer: the built-in
prefect.deployments.steps.git_clone
step does not let you choose a destination folder. It clones into a managed working directory. If you need a specific path (like
C:/user/user/mydir
), use a shell step to clone/pull to that path, then set the working directory. Two common patterns: 1) Clone and run from a subdirectory inside the repo (most common)
Copy code
pull:
  - prefect.deployments.steps.git_clone:
      repository: <https://github.com/owner/repo.git>
      branch: main
      # optional: only bring over specific subfolders; NOT a destination path
      # directories:
      #   - flows
  - prefect.deployments.steps.set_working_directory:
      # relative to the cloned repo root
      directory: flows/my_flow
2) Clone/pull into an explicit absolute path (e.g., Windows), then set it as the working dir
Copy code
pull:
  - prefect.deployments.steps.run_shell_script:
      script: |
        REM Pull if the repo already exists, otherwise clone
        if exist "C:\user\user\mydir\.git" (
          git -C "C:\user\user\mydir" pull
        ) else (
          git clone <https://github.com/owner/repo.git> "C:\user\user\mydir"
        )
  - prefect.deployments.steps.set_working_directory:
      directory: C:\user\user\mydir
Notes: - The
directories
key on
git_clone
is for selecting subdirectories from the repo to include; it is not a destination path. - If your worker runs in Docker or Kubernetes, an absolute Windows path won’t exist inside the container—use relative paths within the container’s filesystem instead. Docs for reference: - prefect.yaml guide - git_clone and set_working_directory API If you can share whether your worker is running on Windows directly vs. in Docker/Kubernetes, I can tailor the exact config.
r
@Marvin what happens if you clone the same repo at each runs? would it overwrite or create a new directory
m
thought for 187.3 seconds
For the built-in
prefect.deployments.steps.git_clone
in Prefect 3.x, each run clones into a fresh, managed directory. It does not overwrite a previous clone and it does not reuse the same path across runs. The step returns the clone path as
directory
, which you can reference like
{{ clone.directory }}
in subsequent steps. If you want to reuse a fixed location (and avoid recloning), you’ll need to manage it yourself with a shell step, e.g., detect the folder and run
git pull
if it exists or
git clone
if it doesn’t. In that pattern, repeated runs will reuse that directory, and your script dictates whether it updates or overwrites. References: - git_clone and set_working_directory API - prefect.yaml guide