I’m looking to set up an automated way to prune old/test flows in a project. I found the GraphQL endpoint to remove flows, but to determine old flows vs current flows I was thinking to:
1. Pipe the output of
prefect register -p project
to get the names of the current flows that would have been registered
2. Get the IDs of the flows through the GraphQL API
3. Delete flow IDs that that do not have names from step 1
Is there a more sensible way to get the “current flows” in a project beside the
prefect register
trick?
k
Kevin Kho
05/03/2022, 9:14 PM
How do you define a
current
flow? Are you giving a list?
m
Matthew Roeschke
05/03/2022, 9:17 PM
We have a project
main
that has flows registered from a git branch
main
via a CI/CD job.
Since our flows in main are changing, “current” would mean any flow that is defined at the HEAD of the
main
branch.
k
Kevin Kho
05/03/2022, 9:18 PM
What do you think of just deleting the project and re-registering the flows?
m
Matthew Roeschke
05/03/2022, 9:19 PM
That would remove the flow history though correct?
k
Kevin Kho
05/03/2022, 9:19 PM
Ah yeah that’s true
Kevin Kho
05/03/2022, 9:20 PM
Ok so number 2 and 3 in your list are pretty doable. But 1 is the hard part. Does piping work for you? I think what you have is a good approach
m
Matthew Roeschke
05/03/2022, 9:23 PM
Yeah piping is an okay workaround. Just curious to find out if there’s a programmatic way to determine a diff of flows that were registered in one call but not another
k
Kevin Kho
05/03/2022, 9:25 PM
Yeah not built-in. You’d have to manipulate the thing you are piping
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.