Is there a way to process the results of a task as they complete? For instance, I have many tasks doing data processing on chunks of a dataframe and I want to aggregate the results of the tasks into an array as they complete. Because I want to operate on the aggregated results, I want this operation to be in the flow and not in the task itself. The closest I can find is this post from @Michael Adkins a few years ago but I’m wondering if anything has changed.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
