<@ULVA73B9P> If I want the target runtime environm...
# ask-community
p
@Marvin If I want the target runtime environment to point at different git branches (e.g. prod points at 'main' and test points at 'dev') would that also be possible without creating an additional deployment?
m
thought for 157.0 seconds
Short answer: no. In Prefect 3.x, the Git ref (branch/tag/commit) is part of the deployment’s pull steps and is fixed when the deployment is created. It can’t be changed at run time via parameters, env vars, or work pool job variables. Recommended approach: - Create separate deployments pointing at each branch (e.g., one for dev -> dev, one for prod -> main). They can share the same flow/entrypoint and differ only in source ref, schedules, params, tags, or work pool. Example:
Copy code
from prefect import flow
from prefect.runner.storage import GitRepository

@flow
def my_flow():
    ...

# Dev
flow.from_source(
    source=GitRepository(url="<https://github.com/org/repo.git>", branch="dev"),
    entrypoint="flows.py:my_flow",
).deploy(name="my-flow-dev", work_pool_name="my-pool")

# Prod (optionally pin to a commit for reproducibility)
flow.from_source(
    source=GitRepository(url="<https://github.com/org/repo.git>", branch="main"),
    entrypoint="flows.py:my_flow",
).deploy(name="my-flow-prod", work_pool_name="my-pool")
If you want to avoid duplication in code, use CI/CD to update each deployment with the correct branch. You can template the branch in prefect.yaml with env vars or Prefect variables, but that’s evaluated when you create/update the deployment—not at run time—so it still results in separate deployment definitions. If helpful, I can sketch a minimal prefect.yaml or CI job that updates both deployments cleanly.
p
@Marvin are there any plans to accommodate this flexibility in a single deployment in future?
m
thought for 171.3 seconds
Short answer: I’m not aware of any public plans to enable switching Git branches at run time within a single deployment. I checked recent issues/discussions and didn’t find a roadmap item for this. If this flexibility would help your team, I recommend opening an Ideas discussion so the team and community can weigh in: - Prefect Discussions: Ideas Today’s approaches: - Use separate deployments per environment/branch (recommended) - Advanced: create a custom pull step that selects the branch at job start (e.g., from an env var or Prefect Variable) and performs the clone itself If you’d like, I can draft an Ideas post summarizing the use case and proposed behavior and share it here for review before submitting.
p
@Marvin would moving away from using GitRepository as the source for a deployment enable something like this to be done?
m
thought for 117.0 seconds
Short answer: Simply not using GitRepository doesn’t, by itself, unlock run‑time branch switching. The limitation comes from when code is fetched/imported (pull steps happen before your flow code loads) and how deployment config is baked. However, you can achieve single‑deployment, run‑time selection by taking control of the pull logic instead of relying on GitRepository’s fixed branch. Common patterns: - Custom pull step (recommended for this use case) - Write a small step that picks a branch at run time (from an env var or a Prefect Variable), then performs the clone. - Because this step runs before your flow is imported, it can prepare the working directory with the desired branch for that run. Example: prefect.yaml
Copy code
pull:
  - my_steps.choose_branch:
      id: pick
      default: main
  - prefect.deployments.steps.git_clone:
      repository: <https://github.com/org/repo.git>
      branch: "{{ pick.branch }}"
      credentials: "{{ prefect.blocks.github-credentials.my-token }}"
  - prefect.deployments.steps.set_working_directory:
      directory: repo
deployments:
  - name: my-single-deployment
    entrypoint: flows.py:my_flow
    work_pool:
      name: my-pool
    # Optionally define job_variables with env to drive the branch at runtime
    job_variables:
      env:
        GIT_BRANCH: main
my_steps.py
Copy code
async def choose_branch(default="main") -> dict:
    import os
    branch = os.getenv("GIT_BRANCH", default)

    # Optional: allow a Prefect Variable to override
    try:
        from prefect.client.orchestration import get_client
        client = await get_client()
        var = await client.read_variable_by_name(name="branch_override")
        if var and getattr(var, "value", None):
            branch = var.value
    except Exception:
        # Don’t fail the pull pipeline if variables aren’t available
        pass

    return {"branch": branch}
Ways to drive the branch per run: - Set an environment variable per run (e.g., via job_variables on the deployment/work pool, or via your run-creation automation). - Flip a Prefect Variable (e.g.,
branch_override
) without redeploying. Other workable approaches if you don’t want Git: - Remote storage artifact selection: publish per-branch bundles (S3/GCS/Azure) and use a custom step (or step output chaining) to pick the path/key at run time before
pull_from_remote_storage
. - Container image selection: build images per branch and choose the image tag via job_variables. This selects code at runtime indirectly, if your process can override job variables on run submission. Trade-offs to be aware of: - Reproducibility: a single deployment that dynamically pulls branches makes “what code ran?” depend on external state (env var/variable), so pinning exact versions per run is harder. - Governance: multiple deployments remain the clearest, most auditable path for prod vs. test. If you share your worker type (Kubernetes/Docker/process), how you launch runs (UI, API, CI), and where you’d prefer to set the branch (env var, Prefect Variable, etc.), I can tailor a minimal working example for your setup.