@Marvin #C04DZJC94DC remove pending flow runs from prefect based on time filter cli?

It appears that there are no issues directly related to removing pending flow runs based on a time filter using the Prefect CLI in the GitHub repository. However, if you want to remove pending flow runs based on time, you might have to write a script that uses the Prefect API (or the Prefect CLI) to query for flow runs, filter them by time, and then cancel them. Here's a rough idea of steps you could take: 1. Use the Prefect CLI or API to list all pending flow runs. 2. Filter the list of flow runs based on your time criteria (e.g., runs that were created more than X hours or days ago). 3. Iterate over the filtered list and cancel each pending flow run with the Prefect CLI or API. This isn't an out-of-the-box feature, so you'll have to implement the logic yourself. The Prefect documentation might have more details on how to interact with the API or CLI for flow runs management. I could give you a more cheerful outlook, but as you'd expect, I won't. Instead, what I will do is wish you the best of luck in this endeavor. If you have specific questions about the Prefect API, CLI commands, or need assistance constructing the script, feel free to ask.

@Marvin share the cli command with sample filter

Sure, I might not be all sunshine and rainbows, but I can still provide some guidance. Here's how you might craft a CLI command to list and then remove pending flow runs based on a time filter. This example uses the Prefect CLI along with

jq

, a command-line JSON processor, to parse and filter flow runs by time. First, you need to list all flow runs with their states and created times:

bash
prefect flow-run list --state=pending --json --no-pager

Next, use

jq

to parse and filter these by a specific creation time threshold. Here's an example of how you could do that:

bash
prefect flow-run list --state=pending --json --no-pager | jq '.[] | select(.created < "2023-04-01T00:00:00Z") | .id'

This

jq

command filters out flow runs that were created before April 1, 2023. You can adapt the timestamp to your needs. After identifying the flow runs, you can loop over them and cancel each one, like this:

bash
for run_id in $(prefect flow-run list --state=pending --json --no-pager | jq -r '.[] | select(.created < "2023-04-01T00:00:00Z") | .id'); do
    prefect flow-run cancel $run_id
done

Please note that you need

jq

installed on your system to run these commands, and that

--no-pager

option disables pagination for the output. Adjust the date in the

select

filter to suit your time-based criteria, and make sure to adjust the state if you need to filter runs with a different status. Also please understand that mishandling CLI commands can lead to canceling the wrong flow runs, so proceed with caution and maybe test with one ID first. But knowing my luck, you probably knew that already, didn’t you?

Hey Nurul! You should be able to refactor this example which uses the Prefect API to remove runs that match a certain set of states. It should be possible to add an additional filter for time as well.