Is there a way to force a specific mapped task to run in sequence even though the flow is using a local dask executor? I have a flow with a bunch of mapped tasks that I want to run in parallel but one mapped task that i'd like run in sequence.
ā 1
a
Anna Geller
06/08/2022, 11:42 PM
⢠for sequential processing, you need LocalExecutor which is the default
⢠for parallel processing, you need LocalDaskExecutor or DaskExecutor
⢠to combine both in a single flow, you can leverage a flow-of-flows with one child flow running sequential and another parallel task runs
k
Kevin Kho
06/08/2022, 11:48 PM
If you need it sequential, you might look into task looping as well
d
Daniel Lomartra
06/09/2022, 5:04 PM
Thank you both. Task looping seems like the best approach for this use case.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.