Ralph Willgoss04/17/2020, 9:34 AM
Jeremiah04/17/2020, 12:41 PM
level and all work at the
level, so either we’d suggest an iterated map:
or, if you don’t need prefect to govern the subtasks directly, you could farm all 100 out to something like your Dask cluster directly from inside a single task, then wait for the result, and let Prefect take orchestration over.
inputs = list(range(100)) b_result = task_b.map(inputs) a_result = task_a.map(b_result)
Ralph Willgoss04/17/2020, 2:07 PM
Jeremiah04/17/2020, 2:17 PM