Mansour Zayer
07/28/2022, 3:28 PMA -> B (trigger: A finished) -> C (trigger: B successful)
I want C
to ignore what happens to A
, but only run if B
is successful. B
will run after A
is finished (no matter fail or success)
I looked into state and state_handlers, seemed too complicated for such a (seemingly) simple task.
edit: In other words, I don't want the trigger function to aggregate ALL the upstream tasks. Just consider the task that I've explicitly set as upstream
ThanksNate
07/28/2022, 3:49 PMfrom prefect import task, Flow
from prefect.triggers import all_successful, all_finished
@task
def task_A():
raise AssertionError
@task(trigger=all_finished)
def task_B():
return 'something'
@task(trigger=all_successful)
def task_C():
return 'something'
with Flow('My Flow') as flow:
result_A = task_A()
result_B = task_B(upstream_tasks=[result_A])
result_C = task_C(upstream_tasks=[result_B])
if __name__ == "__main__":
flow.run()
Mansour Zayer
07/28/2022, 3:55 PMNate
07/28/2022, 4:16 PM❯ python triggers.py
[2022-07-28 10:48:34-0500] INFO - prefect.FlowRunner | Beginning Flow run for 'My Flow'
[2022-07-28 10:48:34-0500] INFO - prefect.TaskRunner | Task 'task_A': Starting task run...
[2022-07-28 10:48:34-0500] ERROR - prefect.TaskRunner | Task 'task_A': Exception encountered during task execution!
Traceback (most recent call last):
File "triggers.py", line 6, in task_A
raise AssertionError
AssertionError
[2022-07-28 10:48:34-0500] INFO - prefect.TaskRunner | Task 'task_A': Finished task run for task with final state: 'Failed'
[2022-07-28 10:48:34-0500] INFO - prefect.TaskRunner | Task 'task_B': Starting task run...
[2022-07-28 10:48:34-0500] INFO - prefect.TaskRunner | Task 'task_B': Finished task run for task with final state: 'Success'
[2022-07-28 10:48:34-0500] INFO - prefect.TaskRunner | Task 'task_C': Starting task run...
[2022-07-28 10:48:34-0500] INFO - prefect.TaskRunner | Task 'task_C': Finished task run for task with final state: 'Success'
[2022-07-28 10:48:34-0500] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded
and from your original description
means that A is necessarily an upstream task of B, even if B doesn't require a successful state from A to runwill run afterB
is finished (no matter fail or success)A