Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

image.png

Hi,

I have some Automations set in Prefect Cloud . Where I've triggered to cancel specific flow when it's a running state for an hour. However I had an instance where this flow was still in the 'running' state and didn't get cancelled for over 5 hours and I had to manually cancel it.

Hey <@U03BS2HBT7S> -- Linking a resolution <https://prefect-community.slack.com/archives/CL09KU1K7/p1692799478154349|here>.  *TLDR:* This is because of the "stays in statement" in combination with the chosen events.

*This is what your automation says:*
"If a flow run emits any one of the `Resuming/Running/Pending` events, and then doesn't emit another event for the next hour, cancel it..."

Chris explains, in the thread I have attached, that "the problem here comes from including both `Pending` and `Running` in the same automation: most flow runs generally go from `Pending` to `Running` quite quickly, so the `Running` event "clears out" the automation after it started from the `Pending` event."

so then just to makes sure the automation should only have running in it?

I also have a custom automation where If there's two lates to notify us. But i also haven't gotten that notification. The script runs every 10 minutes

```{
  "match": {
    "prefect.resource.id": "prefect.flow-run.*"
  },
  "match_related": {
    "prefect.resource.id": [
      "prefect.tag.ingest"
    ],
    "prefect.resource.role": "tag"
  },
  "after": [],
  "expect": [
    "prefect.flow-run.Late"
  ],
  "for_each": [
    "prefect.resource.id"
  ],
  "posture": "Reactive",
  "threshold": 2,
  "within": 2400
}```

Is this a separate automation you've created or is this what you have in the "actions" tab of the automation?

What's the schedule you're currently using? 10 minutes after a run is completed, or every 10 minutes?

Thanks for answering all the questions - just want to make sure I have the full understanding!

the current schedule is that every ten minutes the flow is run. it typically runs in less than 10 minutes.

i'm trying to get it to send a notification if there are two 'late' flows in a row of the flow

Where are you setting the tag on the flow? Can you send the exact value of the tag that you are setting?