Hi All, really interested in Prefect and so far enjoyed playing with it. My questions is around tasks calling other tasks. Can you pass the flow context into other tasks, so that you can set dependencies or perform map functions? Or is it by design that all task orchestration must happen at the top level within the flow context? TaskA does 100 computations, where each computation needs info from TaskB which itself does 100 computations. I'm wanting to take advantage of Prefect distributed nature of tasks out of the box. So i can spin a cluster up and get at least 100 computations all happening on their own nodes.

Hi @Ralph Willgoss - at this time Prefect expects all orchestration to happen at the

Flow

level and all work at the

Task

level, so either we’d suggest an iterated map:

inputs = list(range(100))
b_result = task_b.map(inputs)
a_result = task_a.map(b_result)

or, if you don’t need prefect to govern the subtasks directly, you could farm all 100 out to something like your Dask cluster directly from inside a single task, then wait for the result, and let Prefect take orchestration over.

yeah exactly what i thought. thanks very much as this is how I will treat it

Great!