Krapi Shah

    Krapi Shah

    1 year ago
    Hi all, Is there any automation to send alerts in case of late flow runs? I have currently set an alert for job that does not start for 100 seconds, but this only triggers after it has been scheduled, In case the flow schedule gets delayed due to any reason like agent being down, there is no alert. Any help appreciated! Thanks
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Krapi Shah, there is an automation that says
    when flow does not start after XX seconds, do this
    . Is this the one you tried?
    Krapi Shah

    Krapi Shah

    1 year ago
    yes @Kevin Kho
    Kevin Kho

    Kevin Kho

    1 year ago
    Will ask the team about this
    Jenny

    Jenny

    1 year ago
    Hi @Krapi Shah - Kevin's suggestion is correct - you can use the
    when flow does not start....
    automation. A flow run will enter a scheduled state whether an agent is running or not - in fact flows being stuck in a scheduled state because they couldn't find a suitable agent that is one of the reasons we added this automation. Are you finding that your flows don't enter a scheduled state?
    Krapi Shah

    Krapi Shah

    1 year ago
    @Jenny Out agent was down and hence the flow was added to late runs queue. I already had an alert when flow does not start after 100 seconds, do this, but didnt receive any alerts.
    Jenny

    Jenny

    1 year ago
    Hmmm... has the alert worked perviously for you? Did you get any messages in your notifications tab on the dashboard?
    Krapi Shah

    Krapi Shah

    1 year ago
    No messages in the notifications tab. Not really, faced the scenario for the first time where agent went down
    Jenny

    Jenny

    1 year ago
    No worries. We can use the interactive API to test your action and see if we can get any more info. First let's find the action ID by running:
    query {
      action {
        id
        name
        action_type
      }
    }
    Then you can test the action by using:
    mutation {
      test_action (input: {action_id: ""}){
        success
      }
    }
    Krapi Shah

    Krapi Shah

    1 year ago
    okay..giving it a try
    So we are using Pager Duty and I do see an alert created in PagerDuty on test_action. However, the results also show error code 202 ..Internal server error
    Jenny

    Jenny

    1 year ago
    Hmmm... I'm not an expert on Pager Duty so not sure where the 202 would come from but I see that when I test my pager duty action. However I can verify that I can run a flow with a
    when flow does not start....
    automation with a pager duty action and I get a notification in my pager duty account ok if a flow run is late. For your automation can you check that it is connected to the correct flow? You should be able to see automations connected to a particular flow on the flow page.
    Krapi Shah

    Krapi Shah

    1 year ago
    got it. Thanks @Jenny I will have a look on my end for configs
    Jenny

    Jenny

    1 year ago
    Hi @Krapi Shah - did you have any luck figuring this out?
    Krapi Shah

    Krapi Shah

    1 year ago
    Hi @Jenny I checked the configs and that automation is connected to the correct flow. Also started an arbitrary job with an invalid label so as to make sure that prefect - PD alerting worked. So all settings seem to be in place as expected. But haven't been able to figure out why we didn't receive alert in this particular case. I want to make sure that I have used the correct automation when flow does not start.... if a flow results in a late run irrespective of what maybe causing it.
    Jenny

    Jenny

    1 year ago
    Hmmm.... if it has a late run it should be triggered. Let me dig into this a bit further.
    Hi @Krapi Shah - I have a theory. Do you know if the schedule was already set (i.e. the run was already scheduled) when you created the automation?
    Krapi Shah

    Krapi Shah

    1 year ago
    no... the automation was created earlier than the scheduled run .... almost 2 days prior the issue actually
    Jenny

    Jenny

    1 year ago
    Ah sorry the wording isn't clear there. Was the schedule already set/created? As background, if you create a schedule ahead of a flow SLA automation, the automation does not apply to already scheduled runs. (That's not the same as the run "starting"/entering a scheduled state.). Improving that experience is on our radar but for now toggling the schedule should make sure you get alerts for all future runs.
    Krapi Shah

    Krapi Shah

    1 year ago
    Ahh.... yeah.. thats could be the case. I will toggle the schedule. But we got this alert for other jobs when agent went down. So safe to resolve this. Thanks for the help !!