Hi all, this is a multipart question.
For context, we are building a large Flow that has multiple components that are dependent on one another. For example we have 5 Flows A B C D E. D is dependent on A+B, E on D etc... Certain Flows are also dependent on external data sources from databases or APIs. We are implementing a retry system and a notifications system for these Flows.
1. Is there a way to limit the slack notification to only send on the LAST retry attempt if it fails?
2. For retries, is it possible to set up a retry for only specific errors? For example, if a query times out/connection fails we retry else if there is a syntax error in the query don not retry.
3. Can we send a message for certain error for example an unexpected empty dataframe but still for the flow to complete?
4. Can we trigger a retry of a flow from a different starting point? Let's say there is a failure in the 4th task but we want to retry the run from task 2?
Thank you for any insights on any of these!