Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
pacc-clearcover-june-12-2023
pacc-may-31-2023
ppcc-may-16-2023
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Title
m
Madison Schott
07/20/2022, 3:34 PMHi! My pipeline was successful except for this dbt freshness test so the result flow was green. Usually I receive a notification if a task fails and if the pipeline run is successful and found no alerts. Any ideas on why I wouldn't be receiving either of those?
for example this is what one of the success messages used to look like
k
Kevin Kho
07/20/2022, 4:15 PMMaybe your alerts are on the Flow level and not task level?
m
Madison Schott
07/20/2022, 4:18 PMHow do I check that?
k
Kevin Kho
07/20/2022, 4:18 PMAre you using state handlers or Automations?
m
Madison Schott
07/20/2022, 4:20 PMState handlers I believe
k
Kevin Kho
07/20/2022, 4:23 PMCan I see how you attached it?
m
Madison Schott
07/20/2022, 5:03 PMI used the UI to set it up
k
Kevin Kho
07/20/2022, 5:05 PMI think that is on the Flow level and not task so you’d need to fail the flow is this task fails for that automation to trigger
m
Madison Schott
07/20/2022, 5:14 PMDo you know why the flow doesn't fail if that task fails?
Is it because it has no dependencies?
k
Kevin Kho
07/20/2022, 5:17 PMSo by default, I think the last task is a reference task, but you can set them also. So if something in the middle fails but the last tasks end successfully, it will be labeled successful
m
Madison Schott
07/21/2022, 1:59 PMGot it- thanks! I’ll look into this today and let you know if I have any questions.
In our case, we make a simple tweak at the end of our script to tell this flow to use the run_spark_job task as its reference task:
flow.set_reference_tasks([submitted])is this
[submitted]run_spark_joboh wait never mind I think I see what the docs are referring to now, nvm!
k
Kevin Kho
07/21/2022, 8:49 PMSounds good!
m
Madison Schott
07/21/2022, 8:51 PMis it possible to set that to all_successful with the exception of one task?
k
Kevin Kho
07/21/2022, 8:53 PMI don’t think there is an easy way for that other than using python to manipulate a list of your tasks