Ben Fogelson
11/03/2020, 10:19 PMapply_map with functions that create parameters. Here’s a minimal example:
from prefect import Flow, Parameter, unmapped, apply_map
def func(x, y):
z = Parameter('z')
w = x + y + z
return w
with Flow('flow') as flow:
x = Parameter('x')
y = Parameter('y')
w = apply_map(func, x, y)
---------------------------------------------------------------------------
ValueError: Parameters must be root tasks and can not have upstream dependencies.
The error comes when apply_map tries to add edges from the root tasks of the subgraph to the argument tasks passed to apply_map. This fails when it tries to make an edge from x to z since z is a parameter.
This error seems to preclude ever using apply_map with a function that creates a parameter. I’m wondering if that constraint is necessary. My intuition is that if apply_map sees a parameter that is in the subflow but not in the parent flow, it should assume that any edges from that parameter are unmapped edges that should be added to the parent flow. Does this seem right?