Adrian Fernandez
01/17/2023, 10:34 PMMappingLengthMismatch
error while doing this (more details on the thread) and I have the impression that kwargs can't be unmapped
, neither explicitly nor implicitly. My current work around is to wrap the desired task (A) with another task that doesn't take kwargs and call task a as a function task_a.fn()
.Adrian Fernandez
01/17/2023, 10:37 PMPython 3.8.16
prefect 2.7.8
Adrian Fernandez
01/17/2023, 10:38 PMadd_xy()
without kwargs
from prefect import task, flow, unmapped
@task
def add_xy(x: int, y: int, **kwargs):
print("Kwargs:", kwargs)
return x + y
@flow
def my_flow():
results = add_xy.map(x=[1, 2], y=unmapped(3))
print(my_flow())
# MappingLengthMismatch: Received iterable parameters with different lengths.
# Parameters for map must all be the same length. Got lengths: {'x': 2, 'kwargs': 0}
Adrian Fernandez
01/17/2023, 10:39 PMadd_xy()
with unmapped kwargs
@flow
def my_flow_2():
results = add_xy.map(x=[1, 2], y=unmapped(3), kwargs=unmapped(None))
print(my_flow_2())
# MappingLengthMismatch: Received iterable parameters with different lengths.
# Parameters for map must all be the same length. Got lengths: {'x': 2, 'kwargs': 1}
Adrian Fernandez
01/17/2023, 10:40 PM@task
def add_wrapper(x: int, y: int):
return add_xy.fn(x=x, y=y)
@flow
def my_flow_3():
results = add_wrapper.map(x=[1, 2], y=unmapped(3))
print(results)
print(my_flow_3())
# [Completed(message=None, type=COMPLETED, result=4), Completed(message=None, type=COMPLETED, result=5)]
Adrian Fernandez
01/17/2023, 10:41 PMZanie
Zanie
Adrian Fernandez
01/17/2023, 10:47 PMAdrian Fernandez
01/17/2023, 10:47 PMBen Ayers-Glassey
01/17/2023, 10:58 PMinspect.signature
be used for this?
import inspect
def some_func(x, /, posonly, *args, kwonly, **kwargs): ...
sig = inspect.signature(some_func)
assert sig.parameters['kwargs'].kind == inspect.Parameter.VAR_KEYWORD
for param in sig.parameters.values(): print(f"{param.name}: {param.kind}")
^ That outputs:
x: POSITIONAL_ONLY
posonly: POSITIONAL_OR_KEYWORD
args: VAR_POSITIONAL
kwonly: KEYWORD_ONLY
kwargs: VAR_KEYWORD
Ben Ayers-Glassey
01/17/2023, 10:59 PMZanie
Ben Ayers-Glassey
01/17/2023, 11:03 PMkwargs
dict itself is being checked for iterability, because I believe this error occurs even if you don't pass any kwargs.
So ideally, you'd want the values of the dict to be checked for iterability (or for whether they were marked with unmapped
).
But certainly an empty dict should be allowed!
Otherwise, AFAICT you can't easily write a task which wraps another one (unless you explicitly write out all its parameters in the wrapping function's definition, and explicitly pass them all along).Zanie
Ben Ayers-Glassey
01/17/2023, 11:19 PMZanie