https://prefect.io logo
Title
n

Noam Banay

11/01/2022, 4:26 PM
Hey, I have a flow that is triggered by an API call from an external process. Sometimes, the process fails, and the API call doesn’t send, meaning the flow is not triggered. Is there a way to monitor those delays? I mean - If the flow doesn’t triggered by 8 am, is there a way to monitor it and send a Slack notification or an email? Thanks. @Taylor Curran
1
a

Anna Geller

11/01/2022, 4:44 PM
Hi @Noam Banay, great question! We definitely want to support that pattern, and this is something we are actively working on as part of the Observability feature that will allow you to configure custom SLA exactly as you described.
:upvote: 1
Could you tell us more about the external process that triggers that flow run? perhaps for now we could tackle it on this side until the SLA feature is available
❤️ 1
n

Noam Banay

11/01/2022, 5:26 PM
Thanks @Anna Geller! We are using Rivery as part of our pipeline. Each process (or ‘river’) update a table in our data warehouse (BQ). Those tables are in use in different ETLs queries. So after a river is completed successfully , a REST API call is send to trigger the relevant flow that trigger a DBT job.
a

Anna Geller

11/01/2022, 6:43 PM
in that case, you could solve it by leveraging dbt freshness check -- this would check if the BQ table was updated and only then trigger the dbt run https://docs.getdbt.com/reference/resource-properties/freshness you could error if not fresh and use Prefect flow or task-level retries to retry this run until the freshness check passes (not saying it's an ideal solution, but it would be a pragmatic poor man's SLA)
🙏 1