Ben Fogelson
11/03/2020, 10:19 PMapply_map
with functions that create parameters. Here’s a minimal example:
from prefect import Flow, Parameter, unmapped, apply_map
def func(x, y):
z = Parameter('z')
w = x + y + z
return w
with Flow('flow') as flow:
x = Parameter('x')
y = Parameter('y')
w = apply_map(func, x, y)
---------------------------------------------------------------------------
ValueError: Parameters must be root tasks and can not have upstream dependencies.
The error comes when apply_map
tries to add edges from the root tasks of the subgraph to the argument tasks passed to apply_map
. This fails when it tries to make an edge from x
to z
since z
is a parameter.
This error seems to preclude ever using apply_map
with a function that creates a parameter. I’m wondering if that constraint is necessary. My intuition is that if apply_map
sees a parameter that is in the subflow but not in the parent flow, it should assume that any edges from that parameter are unmapped edges that should be added to the parent flow. Does this seem right?