Hello, Do all mapped tasks have to be completed before dependant mapped tasks can be run.
Let's say you have a flow that has these tasks A -> B-> C.
Tasks B and C are mapped tasks. Is it possible for B to execute right after an individual run of A? Before starting another execution of A
Say Task A adds 10 to input and print the value
Task B Adds 100 to the input and prints the value.
the expected output for the input of task A [1,2,3] would be:
11
111
12
112
13
113
I know that the actual output could be in any order.
k
Kevin Kho
02/14/2022, 9:43 PM
The DaskExecutor prefers depth-first execution, but it can’t be forced that way. On LocalExecutor, it is single threaded so there really isn’t a difference
p
Peter Peter
02/15/2022, 11:03 AM
Thanks I will try to see if I can achieve those results with the DaskExecutor.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.