https://prefect.io
Join Slack
good morning and happy new year! Is it possible to have a task that runs if: 1. some upstream tasks...
# ask-community
k
good morning and happy new year! Is it possible to have a task that runs if: 1. some upstream tasks are successful AND 2. some upstream tasks are successful or failed (just completed)
k
Hey @Kevin Weiler, it sounds like if you have A and B as upstream to C, you want C to fire off if either A or B is done already. Maybe you can try the 
any_successful
trigger. For the second I think you can use an 
all_finished
trigger.
k
thanks @Kevin Kho - I think what I want is for C to trigger if A is successful AND if B finishes. But I wouldn’t want a situation where C triggers if A fails
sort-of like strict and non-strict dependencies
k
I don’t think that can be done with triggers. It might be possible with conditionals and using 
case
, but not sure right now.
k
maybe an intermediate node for the non-strict dependencies
k
That came across my mind as well, but still not sure. 😅. Either way I suspect it can’t be done with a trigger alone at least
k
so, D (the last node now, previously C) would depend on A and C. C would have an 
all_finished
trigger and depend on B
k
i can see that yep
k
thanks for helping me think through it
k
Of course!
k
i guess the conditional thing (not sure if this is what you mean) - but (back to the original example) Task C could have an 
all_finished
trigger and then explicitly inspect the exit status of A - then fail if it is failed
k
You might be able to couple C with A
Copy code
with Flow(...) as flow:
    b = task_B()
    a = task_A()
    with case(a, value):
        c = task_c(upstream_tasks=[b])
or something like that?
k
ah yeah - very interesting
I think that’s functionally equivalent to explicitly inspecting the output of A, but it has the advantage of being expressed in the flow instead of the task
that sound right?
k
I believe so yeah
k
I think in my case it might not matter because I’ve written an abstraction on top of prefect for use at my company. All these tasks are generated from a yaml file anyway. I’m going to try both though
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by