Igor Morgunov
09/06/2022, 4:57 PM@task()
def task1():
# do stuff
@task()
def task2():
# do other stuff
ids = ['aaa', 'bbb', 'ccc']
for id in ids:
x = task1()
y = task2(upstream_tasks=[x])
flow.run(executor=LocalDaskExecutor)
My problem is that task2() fires when the first instance of task1() completes successfully - I need task2() to fire only once all of task1() instances have completed - what am I doing wrong here?Barada Sahu
09/06/2022, 5:08 PMwait_for=[x]Nate
09/06/2022, 6:38 PMtask1.map(ids) instead of a for loop and you need to define your flow as a `with Flow() as flow:`context manager
if prefect 2, there is no longer a concept of an Executor , you'd want to use the DaskTaskRunner
wait_for is indeed what you want if prefect 2, and upstream_tasks is what you want for prefect 1.Igor Morgunov
09/07/2022, 8:40 AM