Vadym Dytyniak
11/30/2022, 2:46 PMl1 = task1.map([1, 2, 3])
l2 = task2.map(l1)
task3.map(l1=l1, l2=l2)
Prefect does not guarantee that in task3
list l1
elements will be passed with corresponding l2
elements?Anna Geller
Vadym Dytyniak
11/30/2022, 4:29 PMAnna Geller
from prefect import flow
from prefect_dask import DaskTaskRunner
from flows.parallel.tasks import extract, transform, load
@flow(task_runner=DaskTaskRunner())
def dask_flow_map():
numbers = extract.submit()
transformed_numbers = transform.map(numbers)
load.submit(numbers=transformed_numbers)
regardless of which task was submitted first etcVadym Dytyniak
11/30/2022, 4:49 PMAnna Geller