https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I have a deployment that pulls from g...
# ask-marvin
t
@Marvin I have a deployment that pulls from git. Can I pass a git hash as a job variable to get a specific version of the code?
m
✅ thought for 639.1 seconds
n
you'd pass a 
ref
to the pull step, which can be a commit hash
t
sorry, how exactly? I can't find docs for 
ref
. Also, ideally I would be able to use some predefined branch whenever I run but if I pass a hash ideally I'd overwrite that. But that may be too much to ask! @Marvin
m
✅ thought for 603.9 seconds
n
m
Short answer: not via job variables. In Prefect 3.x, job variables control infrastructure (env, resources, image, etc.) but they don’t change the git ref your deployment pulls. To run a specific commit per run, parameterize the pull step and set the value via either a Prefect Variable or an environment variable that’s available to the worker at run time. Option A (recommended): Prefect Variable in pull step - prefect.yaml:
Copy code
pull:
  - prefect.deployments.steps.pull.git_clone:
      id: clone
      repository: <https://github.com/owner/repo.git>
      commit_sha: "{{ prefect.variables.target_sha }}"
      # or: branch: "{{ prefect.variables.target_branch }}"
- Update before the run:
Copy code
prefect variable set target_sha abc123def
Workers evaluate this at run time, so you can change it without redeploying. - Docs: Prefect Variables, prefect.yaml, git_clone pull step Option B: Environment variable placeholder in pull step - prefect.yaml:
Copy code
deployments:
  - name: my-deploy
    job_variables:
      env:
        GIT_SHA: "abc123def"
    pull:
      - prefect.deployments.steps.pull.git_clone:
          repository: <https://github.com/owner/repo.git>
          commit_sha: "{{ $GIT_SHA }}"
Ensure the env var is present in the worker’s runtime environment (job_variables.env or your work pool template). The pull step will use it when the worker executes. - Docs: prefect.yaml, Customize job variables Notes - Passing a commit SHA as a flow parameter won’t work for code retrieval—the code is pulled/imported before parameters are read. - If you prefer pinning at deploy time, set 
commit_sha
on 
GitRepository
when calling 
flow.from_source(...)
(e.g., from an env var in CI); that value is then fixed for that deployment version. - This works the same on Process, Docker, and Kubernetes workers; the git clone happens inside the worker at run time. If you want to choose a commit per run from an external trigger/automation, I can share a tiny script/automation that sets the Prefect Variable and then schedules the run.
n
yea so option A from @Marvin, option B makes no sense because 
env
are runtime env values whereas 
commit_sha: "{{ $GIT_SHA }}"
would template an env var 
$GIT_SHA
on the machine running 
prefect deploy
at deployment time
m
✅ thought for 264.3 seconds
t
OK, this is useful. Thanks a lot!
catjam 1
m
You’re right to prefer Option A. - In Prefect 3.x, the reliable, supported way to choose a git ref at run time is to reference a Prefect Variable in your pull step; it’s evaluated by the worker at run time. - Job variables can’t change the git ref. - Environment placeholders are tricky; they can be expanded during 
prefect deploy
depending on where they’re used, so they’re not a great fit for selecting code at run time. Prefect Variables avoid that ambiguity. Example
Copy code
pull:
  - prefect.deployments.steps.pull.git_clone:
      repository: <https://github.com/owner/repo.git>
      commit_sha: "{{ prefect.variables.target_sha }}"
      # or branch: "{{ prefect.variables.target_branch }}"
Set/change before a run:
Copy code
prefect variable set target_sha abc123def
Relevant docs: - Prefect Variables: Variables - prefect.yaml behavior and pull steps: prefect.yaml - git_clone step API: pull steps API If you want, I can share a tiny script or automation pattern that sets 
target_sha
and triggers the deployment in one go.
Gotcha — in Prefect 3.x there isn’t a 
ref
parameter on 
Flow.from_source(...)
. That was a pattern you might have seen elsewhere; in 3.x you’ll use either: - The Git clone step in a project’s prefect.yaml (CLI), or - The GitRepository storage object with the Python API Both support a default branch and an optional commit SHA; when both are provided the commit SHA wins (we clone the branch then checkout the SHA). Option A — Projects (prefect deploy) Use the built-in git clone step. You can set a default branch and optionally pin to a commit at deploy time via an environment variable. prefect.yaml
Copy code
name: my-project
prefect-version: ">=3.0.0"

pull:
  - prefect.deployments.steps.pull.git_clone:
      repository: "<https://github.com/acme/prefect-flows>"
      branch: "main"                          # default branch
      commit_sha: "{{ env.GIT_SHA | default('') }}"  # optional SHA override
      include_submodules: false

deployments:
  - name: prod
    entrypoint: flows/etl.py:etl
    work_pool:
      name: process-pool
Deploy:
Copy code
# Use default branch
prefect deploy

# Pin to a specific commit for this build
GIT_SHA=abc1234 prefect deploy
Option B — Python API (flow.from_source(...).deploy(...)) Construct a GitRepository and pass it as the source. Set a default branch; optionally set a commit SHA when you want to pin.
Copy code
from prefect import flow
from prefect.runner.storage import GitRepository

repo = GitRepository(
    url="<https://github.com/acme/prefect-flows>",
    branch="main",          # default branch
    commit_sha=None,        # set to a SHA to pin
)

f = flow.from_source(repo, entrypoint="flows/etl.py:etl")

f.deploy(
    name="prod",
    work_pool_name="process-pool",
)
When you want to pin to a specific commit for a new deployment:
Copy code
repo.commit_sha = "abc1234"
f.deploy(name="prod", work_pool_name="process-pool")
Can I override the ref at run time? - Not as a first-class “run flag.” In 3.x the code source (branch/SHA) is part of the deployment’s storage configuration. To change what runs, update or create a deployment that points to the desired SHA/branch (you can do this programmatically and reuse the same deployment name, or create a temporary name per SHA). - With Projects, the 
git_clone
step happens at deploy/build time (not per flow run), so the override via 
GIT_SHA
is also a deploy-time decision. Quick references from the API we used: - 
prefect.deployments.steps.pull.git_clone(repository, branch=None, commit_sha=None, ...)
- 
prefect.runner.storage.GitRepository(url, branch=None, commit_sha=None, ...)
- 
Flow.from_source(source, entrypoint)
(no 
ref
argument) Which path are you using right now — the CLI/project (
prefect deploy
with a prefect.yaml) or the Python API (
flow.from_source(...).deploy(...)
)? I can tailor the exact snippet to your setup.
4 Views
Open in Slack
PreviousNext
Powered by