https://prefect.io
Join Slack
Hey team, I have an simple automation that changes a flow run state to Failed if the flow run stays ...
# ask-community
s
Hey team, I have an simple automation that changes a flow run state to Failed if the flow run stays in a Running state for x hours. This has worked previously. But i have also found this doesn't always trigger, despite having flow runs meet the criteria. Has anyone encountered this as-well?
Also related, I have tried the use of timeout_seconds in the declaration of the flow and/or tasks in my deployment scripts, and it appears to not do anything. Further limiting my options. A quick search in this channel/Google also shows users experiencing the same bug.
FYI i am using version 2.15
b
Hi Sam! How have you configured your automation? Would you mind dropping a screenshot of how the trigger is set up?
As for the timeout_seconds, I did find an issue here that seems related. How long are you setting the timeout seconds for?
s
Hey Bianca, Here's my automation:
image.png
As for the timeout_seconds. That issue looks the same as what i'm experiencing. I tested with 1, 3 and 60 seconds
b
Gotcha, thanks for the context. Just to confirm that the automation isn't firing, have you already inspected the events page in the UI? You should be able to filter by events (
prefect-cloud.automation.*
) and by the resource (the flow or deployment that is breaching the SLA defined in the trigger) to see if the automation is A) triggering and B) the action is actually executing. Curious to see if it's not firing at all, or if the action itself just isn't going through correctly. If it's the action that isn't being applied, it may be worth trying to adjust the action so that it cancels the flow run instead of trying to coerce the state, and send yourself a notification (maybe via email) that a flow run breached the SLA.
If you don't see the trigger or action happening, even after making the suggested changes, please open an issue and include the IDs of the flow runs that didn't trigger the automation, and the automation ID! (all the IDs should be present in the URL in your browser).
s
Yep, In the events page the for the automation there are no events, and i can see a flow run that occurred in that time that should have triggered the automation (i have manually cancelled this flow run). I'll raise an issue.
7 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by