https://prefect.io logo
t

Tom A

09/14/2023, 2:15 PM
Posting my question from the call: What's the right way to remove deployments from a prefect environment via CI/CD processes. Along the same lines, are there any suggestions for promoting blocks between prefect environments (i.e. dev->stage->prod)?
2
d

David Maxson

09/14/2023, 2:20 PM
If you want the prefect.yml file to be "the authoritative source", and removing a flow from that file would result in it being removed from the API server, you could probably do a "prefect deployments delete all" (no idea if that's available) then re-deploy all. Deployments seem to be very lightweight objects, so that shouldn't be too burdensome, but I do wonder if that could have ramifications if a flow is running when you do that, but otherwise you do definitely need to determine how to enumerate which flows should be deleted (not everyone uses a single repo to define all flows/deployments in a workspace)
t

Tom A

09/14/2023, 2:30 PM
Thanks David, I'm curious if that would remove my log history for those flows, which I wouldn't necessarily want to do on each code-push
c

Christopher Boyd

09/14/2023, 2:30 PM
I think this is a really good question, and one i think I need to noodle on a bit for a comprehensive answer. The short of it is that once prefect.yaml deploys your flows, they are stored cloud side in the database. You can delete them out of cloud via UI / API, but then to remove them from prefect.yaml
that’s a good question - let me think on this and get back to you
s

Sylvain Hazard

09/14/2023, 2:31 PM
Having a kind of Terraform-ey workflow where the deployment tool could check the diff between the yaml and what the server/cloud knows about could be pretty cool indeed.
t

Tom A

09/14/2023, 2:31 PM
Thanks Chris, I had an idea at one point where I would build a script to read in the yaml and then use the api to read in current deployments and delete out anything that wasn't listed. Similar thing for blocks (but I just had a custom yaml file with the block names as to not leak anything sensitive). This seemed hacky, so definitely looking forward to hearing the prefect-suggested approach 🙂
s

Sylvain Hazard

09/14/2023, 2:32 PM
It does not feel too hard to script with current tool but having it as an internal approach would likely be a little safer
t

Tom A

09/14/2023, 2:33 PM
Yeah for sure, my script technically worked, but seems iffy to put in prod haha
s

Sylvain Hazard

09/14/2023, 2:34 PM
Isn't putting iffy stuff into prod the basics of software engineering ? 😛
upvote 1
😬 1
d

David Maxson

09/14/2023, 2:40 PM
The terraform mention reminded me that a lot of the CLI operations feel like lightweight Terraform-style commands... but as this thread makes clear, they're not as stateful as terraform. This might be a good reason to add a resource to the prefect terraform provider to handle deployments, then just use TF to add/delete deployments rather than the CLI.