Channels
#prefect-ui
Title
m
Madison Schott
07/20/2022, 3:34 PMHi! My pipeline was successful except for this dbt freshness test so the result flow was green. Usually I receive a notification if a task fails and if the pipeline run is successful and found no alerts. Any ideas on why I wouldn't be receiving either of those?
for example this is what one of the success messages used to look like
k
Kevin Kho
07/20/2022, 4:15 PM
Maybe your alerts are on the Flow level and not task level?
m
Madison Schott
07/20/2022, 4:18 PMHow do I check that?
k
Kevin Kho
07/20/2022, 4:18 PM
Are you using state handlers or Automations?
m
Madison Schott
07/20/2022, 4:20 PMState handlers I believe
k
Kevin Kho
07/20/2022, 4:23 PM
Can I see how you attached it?
m
Madison Schott
07/20/2022, 5:03 PMI used the UI to set it up
k
Kevin Kho
07/20/2022, 5:05 PM
I think that is on the Flow level and not task so you’d need to fail the flow is this task fails for that automation to trigger
m
Madison Schott
07/20/2022, 5:14 PMDo you know why the flow doesn't fail if that task fails?
Is it because it has no dependencies?
k
Kevin Kho
07/20/2022, 5:17 PM
So by default, I think the last task is a reference task, but you can set them also. So if something in the middle fails but the last tasks end successfully, it will be labeled successful
m
Madison Schott
07/21/2022, 1:59 PMGot it- thanks! I’ll look into this today and let you know if I have any questions.
Copy code
In our case, we make a simple tweak at the end of our script to tell this flow to use the run_spark_job task as its reference task:
flow.set_reference_tasks([submitted])is this
[submitted]run_spark_joboh wait never mind I think I see what the docs are referring to now, nvm!
k
Kevin Kho
07/21/2022, 8:49 PM
Sounds good!
m
Madison Schott
07/21/2022, 8:51 PMis it possible to set that to all_successful with the exception of one task?
k
Kevin Kho
07/21/2022, 8:53 PM
I don’t think there is an easy way for that other than using python to manipulate a list of your tasks
3 Views