I have a large flow that I use to query data, transform it, test it, and insert it into a database (with each of these steps as either tasks or sub-flows). My question is what is a best practice regarding:
1. Failing the entire flow if one part of the flow (a sub flow or task) fails? I want the entire flow to fail if the testing discovers an issue with the data (I have custom Pandera data checks setup because our database frequently experiences failures). I've thought about passing an exception to each subsequent task/sub-flow but that seems cumbersome.
2. Emailing details regarding failures. I've setup a custom function to email exceptions if they occur. Should I just insert a try/except for each task/sub-flow? I think that would work, but wasn't sure if there was a better method.