Hi all,
Using prefect cloud here; is there any way of automatically cleaning up old flow-runs, say I don't need anything > 24 hours old? Its starting to clog up a bit and running a script manually feels a bit strange since I don't like running manual stuff -- sort of what I got prefect for in the first place 🙂
Thanks!
n
Nate
06/01/2023, 4:21 PM
hi @Robin Doornekamp
while I don't think there's a first-class feature specifically for cleaning up old flows after some time interval (besides our data retention policy), it sounds like this could be a pretty quick flow to write and schedule if you wanted
probably just:
• read_flow_runs with a flow_run_filter to grab all flow runs older than X
• delete_flow_run by ID for each matching flow run
r
Robin Doornekamp
06/02/2023, 7:09 AM
@Nate Yeah thats what I'm doing; was just curious if there was some configuration outside of the data retention -- but ill just stick with running a flow to clean up the flows then. Thanks for the reply!
n
Nate
06/02/2023, 2:20 PM
okay! out of curiosity, what would be your ideal UX here? I can open an enhancement ticket
r
Robin Doornekamp
06/04/2023, 8:26 PM
Thanks for inquiring; let me think; I've noticed that I have no real purpose to have historic flow runs around for longer than
x
days; they kind of clutter up the view and make the UI a bit slow. I would have maybe expected some kind of configuration; almost on workspace level, for how long these things are retained.
I'm not a UX designer but the first place I looked was if there was some kind of workspace-level settings or deployment settings (because I thought maybe retention is different for different ones).
So ideally for me would be; I can configure a retention time for finished flow runs.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.