When triggering Task A to run, does roughly the following happen? 1. The flow follows the edges to find the subset of tasks it needs for Task A. 2. It has TaskRunner run the tasks, starting from the root tasks, extracting results from their final/finished states and passing along downstream edges. The root tasks are just Parameters, their inputs are accessible ahead of time in

prefect.context["parameters"]

3. You can't easily access non-Param task inputs passed in

run

, because they come from the /final/ states of upstream tasks (and you can't just "wait" for these inputs, because you don't know which object they're from; the final upstream state object hasn't even been created)

In this case, each Task.run() returns a list of Futures (as a workaround of

map

making 100k tasks each doing the same 1 thing, we can instead have 1 task do 100k things, i.e. submit 100k requests to an external slurm-like scheduler we use, which returns us 100k futures).

alternatively

you could implement your own

TaskRunner

class and use it for execution -> this might actually be the best way to go

and this method would just be a new step in the task runner pipeline - you could override one of our internal steps and implement custom logic