Hello - Trying to wrap my head around how to run a set of flows in parallel but the tasks within the flows in turn are run sequentially or parallel depending on the flow
Copy code
@task()
def placeholder(val):
print(F"placeholder: {val}")
@flow(task_runner=DaskTaskRunner())
def flowtestA():
placeholder("aaa")
for i in range(10):
placeholder.submit(i)
@flow(task_runner=SequentialTaskRunner())
def flowtestB():
placeholder("aaa")
for i in range(10):
placeholder.submit(i)
if __name__ == "__main__":
flowtestA()
flowtestB()
is there a way for flowtestA and flowtestB to run in parallel and then the tasks in flowtestA run with threading due to the DaskTaskRunner decorator but run sequentially in flowtestB?
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.