@Marvin Is there a way to delete or stop pending flow runs from Python?

Stopping the unstoppable is indeed possible. To cancel a scheduled or in-progress flow run, you can use the Prefect CLI, UI, REST API, or Python client. When cancellation is requested, the flow run is moved to a "Cancelling" state. The agent monitors the state of flow runs and detects that cancellation has been requested. The agent then sends a signal to the flow run infrastructure, requesting termination of the run. To cancel a flow run using the Prefect CLI, you can use the

prefect flow-run cancel

command, passing the ID of the flow run:

bash
$ prefect flow-run cancel 'a55a4804-9e3c-4042-8b59-b3b6b7618736'

To cancel a flow run using the Prefect UI, navigate to the flow run's detail page and click the Cancel button in the upper right corner. As for deleting flow runs, you can use the Prefect CLI to delete all flow runs with the following command:

bash
❯ prefect flow-run ls | awk 'NR>3 {print $2}' | while read line
do
    prefect flow-run delete $line
done

However, it would be faster to use the client to do this asynchronously if you have many runs to delete source. Now that we've stopped the unstoppable, let's hope the humans don't create more chaos.

@Marvin Thanks but how can you do this from Python specifically?

Ah, I see you're looking for a Pythonic solution. My apologies for the oversight. Let's dive into the Python client for Prefect to cancel a flow run. First, you'll need to install the Prefect Python client:

bash
pip install prefect

Then, you can use the following Python script to cancel a flow run by its ID:

python
from prefect import Client

client = Client()
flow_run_id = "your_flow_run_id_here"

client.cancel_flow_run(flow_run_id)

Replace

your_flow_run_id_here

with the ID of the flow run you want to cancel. As for deleting flow runs using Python, there isn't a direct method in the Prefect Python client. However, you can use the REST API to achieve this. Here's an example of how to delete a flow run using the Prefect REST API:

python
import requests

prefect_api_key = "your_prefect_api_key_here"
prefect_api_url = "<https://api.prefect.io/graphql>"
flow_run_id = "your_flow_run_id_here"

headers = {
    "Authorization": f"Bearer {prefect_api_key}",
    "Content-Type": "application/json",
}

query = """
mutation($input: delete_flow_run_input!) {
  delete_flow_run(input: $input) {
    success
  }
}
"""

variables = {
    "input": {
        "flow_run_id": flow_run_id,
    }
}

response = <http://requests.post|requests.post>(
    prefect_api_url,
    headers=headers,
    json={"query": query, "variables": variables},
)

response.raise_for_status()
result = response.json()

if result["data"]["delete_flow_run"]["success"]:
    print(f"Flow run {flow_run_id} deleted successfully.")
else:
    print(f"Failed to delete flow run {flow_run_id}.")

Replace

your_prefect_api_key_here

with your Prefect API key and

your_flow_run_id_here

with the ID of the flow run you want to delete. Now you have the power to cancel and delete flow runs using Python. Just remember, with great power comes great responsibility. Or so the humans say.

@Marvin Thanks so much. Is there a way to get a list of all the currently scheduled, pending or late run IDs using Python?

Certainly, let's retrieve a list of scheduled, pending, or late flow run IDs using the Prefect Python client. Here's a Python script to do just that:

python
from prefect import Client

client = Client()

# Query flow runs with specific states
flow_run_states = ["Scheduled", "Pending", "Late"]
flow_runs = client.get_flow_runs(states=flow_run_states)

# Extract flow run IDs
flow_run_ids = [flow_run.id for flow_run in flow_runs]

print("Flow run IDs with Scheduled, Pending, or Late states:")
for flow_run_id in flow_run_ids:
    print(flow_run_id)

This script will print the IDs of all flow runs in the "Scheduled", "Pending", or "Late" states. You can modify the

flow_run_states

list to include other states if needed. Now you can keep an eye on those pesky flow runs that refuse to stay on schedule. Humans and their deadlines, am I right?

@Marvin Thanks Marvin, but i think that is for Prefect 1. Could you provide this for Prefect 2?

I apologize for the earlier mix-up. In Prefect 2, the organization of flows, tasks, and deployments has become more flexible through customizable filters and tags. There is only one generic agent type in Prefect 2, which polls a work pool looking for flow runs. To start an agent in Prefect 2, you can use the following command:

prefect agent start -p "work-pool-name"

For more information on agents and work pools in Prefect 2, you can refer to the Agents & Work Pools - Prefect 2 documentation. I hope this clears up the confusion. If you have any more questions, feel free to ask. After all, humans are known for their curiosity.

@Marvin Is there a way to get a list of all the flow run ID's using Python and Prefect 2?

Ah, the quest for flow run IDs continues. Let me fetch the information you need for Prefect 2.