Hi there! General question about testing my deployments on a git branch. I have 5 production deployments up and running. The changes in my git branch affects all 5 flows and I need to deploy and test them all before merging. A naive way of testing would be to create 5 new test deployments in my

prefect.yaml

(with different names to the production deployments). This approach seems inefficient, and does not play nicely with

run_deployment

, which will need the names of these test deployments. Another way of solving this is to override the git branch at runtime (without creating any new deployments). I believe this can be done by overriding

job_variables

when you launch a run. Is this the correct route to go down? My problem feels relatively standard, and yet this solution feels hidden and not well-maintained. I say "hidden" because in the UI you have to select 'custom run' and then 'additional options' to find it. And I say "not well-maintained" because I have encountered several bugs with overriding job variables as acknowledged here and here. I am very curious how other people are testing their deployments inside git branches.

Hey @benorbital - it's a good call, we definitely need a tighter git integration. In the meantime, you can also use environment variables (which are technically a part of job variables but a more transparent mechanism) to set the branch and dynamically resolve it at runtime. So in this case you'd set the environment variable on a per-run basis.

Thanks for that answer Chris! This is the solution I went with in the end, so nice to have it confirmed! Tighter git integration would be great 🙂 Right now, I have to do some manual work to pass these env vars to

run_deployment

in my code, so automating that would be wonderful. I think it's still the case that the github issues I linked to make it tricky to specify environment variables at runtime (especially via the CLI). Overall, I am loving prefect! ❤️ The git integration is the biggest issue i've had so far

Awesome thank you for the feedback -- glad you have a workaround for now, and we're going to be prioritizing many of our integrations (cloud providers + git) in the next few weeks, so I'll be sure to revisit this! For my own understanding, would your ideal use case be something like: open PR to your repo -> trigger a deployment run off that branch -> get some status badge back to github confirming the run was successful? Or is more involved in testing?

Having it built into the CI like that is more like the cherry on the cake for me. The main thing is that when I run

prefect deployment run

, I have some easy way of specifying a git branch. Potentially it would be built into the git clone step that exists i.e this:

pull:
    - prefect.deployments.steps.git_clone:
        repository: <https://gitlab.com/org/my-private-repo.git>
        access_token: "{{ prefect.blocks.secret.my-block-name }}"

The other (admittedly more demanding) ask is that subflows executed with

run_deployment

can inherit this git branch automatically.

It's not unreasonable if we expose branching in a more first class way but I agree it's hard to commit to before wiring things up; either way, this is great feedback that I will definitely incorporate into our plans for integrations!

if we expose branching in a more first class

This would also be great for me, as multi-infrastructure branching is the main reason I started using prefect

Oh interesting, the plot thickens!! Would you be open to chatting through your setup / use case at some point in the next few weeks? I would love to learn a bit more and see if there's a broader pattern lurking in here for which per-run git-branching is one application.

Yeah, very happy to chat about our use-case! For more context: I work at a AI materials science startup with somewhat diverse compute needs, so I imagine it could be a fruitful discussion

yeaaaa that sounds awesome! Here's a link to my calendly (I'm blocked for the next two weeks but my schedule opens back up in June): https://calendly.com/chris-at-prefect/30min-2 and feel free to email me if none of those times work (

<mailto:chris@prefect.io|chris@prefect.io>

)

@Chris White: just FYI, specifying git branches via env vars is not really viable for

run_deployment

, as I discovered here: https://prefect-community.slack.com/archives/CL09KU1K7/p1716576242919739

Oh wow, yea that's no good - I'll see if I can prioritize this issue