Has anyone ever documented (or can share) a release process for prefect flows code? Say, I have a repo that describes a few flows: A, B, and C. I have the code deployed as a docker image + deployment in GCS bucket. I would now like to allow gradual/controlled rollout of the repo code changes (deployment code to GCS, and container image) to each of A, B, C in each of QA and Prod environments. The not-the-most-straightforward way I can think of is: 1. For every commit in the repo, push a docker image with the commit hash as a label and deploy code to e.g.

gs://<bucket>/<commit hash>

2. For each of the cartesian product of Flow ⅹ Env (e.g.

A-prod

,

B-qa

) have: a. A separate storage block pointing to a different subdir of

gs://<bucket>/

b. A docker image label pointing to the image of the same commit. 3. The release to an environment would then include: updating the storage block to point to a different dir + updating the container image tag. Each deployment would also have an

infra_override

pointing it at its specific image tag. A couple of concerns I’m having are: storing all code/image versions (might be a good thing?). Maintaining potentially a large number of storage blocks (one for each of flow x env). Any thoughts?

We use different workspaces for different environments, and docker image tags to separate between development (uses latest tag) and production (uses some v1, v2 etc tag). But if you don’t have the option to use multiple workspaces, a couple of thoughts partially inspired from Prefect 1 where I tried to use projects to separate envs: 1. I use a new, randomly generated code location for every time I register a deployment. It might seem wasteful, but the files are small and after the initial development period they don’t change much. This way, I won’t have to change the storage location on deploy. 2. I also used OS-level env vars a lot. So you can have docker image references, storage locations, block names etc be determined by some

PRREFECT_DEPLOY_TO

variable. If the variable isn’t set, we can assume it is a developer using their local machine, and everything can be resolved to have a

dev_

prefix or something like that. We have a rule that everything in production must be deployed from the main branch, and so we have a github action to register the flows. Here, you can explicitly set the

PREFECT_DEPLOY_TO

variable to

prod

or something like that, and all references to storage blocks etc will be resolved without the

dev_

prefix. 3. We do by default use a single docker image in multiple flows. We can of course build particular docker images for particular flows, but this process is detatched. In prefect 1 we had a 1 flow = 1 image policy, but it took a long time to build and development cycles dragged out. It seems the images<->flows connection is a bit more central to you, so if you do need a particular block to correspond to a particular version of a flow I would look at using a randomly generated string and register a block automatically on deployment and use infra overrides to specify the image so you don’t need a million blocks. But this isn’t something I have tried out, I’m just thinking out loud at this point. Hopefully this helps, although it isn’t a direct answer to your question.