https://prefect.io logo
Title
b

Barada Sahu

09/06/2022, 5:16 PM
If there’s a task failure in an upstream task and I am waiting on it to complete (using
wait_for
), it’s never triggered, rather goes into an undocumented
NotReady
state. To overcome this, I have to set
.result(raise_on_failure=False)
on the upstream task. See images below for states. See screengrabs 👇 - left fails, right works. This seems like a common DAG use-case which should be handled well by prefect, how do we wait on a set of upstream tasks to complete and then perform a cleanup / notification at the end (with an aggregate of results from all upstream tasks).
1
z

Zanie

09/06/2022, 5:17 PM
Thanks for the report!
wait_for
indicates that the upstream needs to complete successfully for the downstream to run.
Perhaps we can allow the upstream to reach any final state.
b

Barada Sahu

09/06/2022, 5:19 PM
Yeah ! that would make a lot more sense. This seems like a pretty common use-case where a downstream task needs to execute after all upstream ones are done - Completed or Failed.
o

Oliver Mannion

09/22/2022, 9:10 AM
+1 we currently use
@task(trigger=any_successful)
in Prefect 1, and are looking for something similar in Prefect 2