https://prefect.io logo
r

Robin

07/25/2023, 1:10 PM
Is there a way to create an automation that triggers when there are
NO flows in certain states for a certain time
? Maybe there is another more elegant solution, but the idea is to shut down certain resources if there are no flows running, pending or late etc. 🙂
1
w

Will Raphaelson

07/25/2023, 3:36 PM
You can use a custom trigger to accomplish this, in particular a trigger woith a proactive posture. so after some terminam flow run state, expect a flow run to enter a scheduled or running state, and if NOT, then fire the automation. you can read more about it here. https://docs.prefect.io/2.11.0/cloud/automations/
r

Robin

07/25/2023, 4:35 PM
OK, I read the documentation but did not yet understand how the proactiveness works
I can poke on it a bit but either way more description and more examples in the docs would be highly appreciated 🙏
Let me know, where I could create a PR for that 🙂
Starting off the following, what do I need to further adapt?
Copy code
{
  "match": {
    "prefect.resource.id": "prefect.flow-run.*"
  },
  "match_related": {},
  "after": [
    "prefect.flow-run.Success"
  ],
  "expect": [
    "prefect.flow-run.Running"
  ],
  "for_each": [
    "prefect.resource.id"
  ],
  "posture": "Proactive",
  "threshold": 1,
  "within": 180
}
Does this automation do the following? • whenever a flow run switches to
Success
• expect another flow to be in running state within 3 minutes • if not: trigger an action
w

Will Raphaelson

07/25/2023, 4:49 PM
i hear you on more examples, i can get a catalogue together in the near term. nice job on the trigger - a few tweaks delete the whole for_each key. this says that it should evaluate the trigger for each flow run to go from success -> running, you actually want to execute the action if any flow run goes into running after a success, not just that flow run.
Copy code
prefect.flow-run.Success
is actually not an event. what you’re looking for is prefect.flow-run.Completed. you can see the recent events in the event feed.
❤️ 1
💡 1
🙌 1
r

Robin

07/26/2023, 6:24 PM
Thanks for the feedback and for collecting more examples! I made the changes you suggested. However, I want the automation to
trigger a flow run, if no other flow switches to the Running state within 3 minutes
Does the following code do that?
Copy code
{
  "match": {
    "prefect.resource.id": "prefect.flow-run.*"
  },
  "match_related": {},
  "after": [
    "prefect.flow-run.Completed"
  ],
  "expect": [
    "prefect.flow-run.Running"
  ],
  "posture": "Proactive",
  "threshold": 1,
  "within": 180
}
w

Will Raphaelson

07/26/2023, 6:27 PM
yep. that looks right to me!
🙏 1
r

Robin

07/26/2023, 6:27 PM
Alright, will "test in production" 😄
the rule just triggered although there are still flows in the running state 🤔
@Will Raphaelson, any idea, how the script can be improved, so that the automation does not trigger, when there are still flows in the running state?
1
w

Will Raphaelson

07/27/2023, 3:00 PM
Sorry, I think I misunderstood the initial requirements. what the automation we discussed will achieve is to execute the action when any flow run emits a completion event, and then no flow run emits a running event within the time period. Other flow runs that were running from before the 3 minute window may continue to run and wouldn’t send a running event (we send a running event when the flow run ENTERs running, not continuously throughout runtime) so the action could fire even with flows running. I think that automations may not actually be the best fit here, in retrospect. Perhaps a flow that polls the /flow_runs endpoint and gets all running flows would be useful. if the flow returns no running flows, than execute a subflow that does what you need the action to do. Does that sound closer?
r

Robin

07/27/2023, 4:20 PM
Yeah, that sounds better!
Any hint on how to do the polling?
Got it running 🙂
🔥 1