https://prefect.io logo
Title
m

Mansour Zayer

03/22/2023, 4:36 PM
Hello. In Prefect2, I have a flow with
**kwargs
for its input parameter like
my_flow (p_1, p_2, **kwargs:dict[str, bool])
. When I run the flow from the UI, I get
Validation of flow parameters failed with error: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 296, in retrieve_flow_then_begin_flow_run
    parameters = flow.validate_parameters(flow_run.parameters)
  File "/usr/local/lib/python3.9/site-packages/prefect/flows.py", line 339, in validate_parameters
    raise ParameterTypeError.from_validation_error(exc) from None
prefect.exceptions.ParameterTypeError: Flow run received invalid parameters:
 - kwargs: value is not a valid dict
the parameter value is:
{
  "kwargs": {
    "p_1": false
  }
}
Am I doing something wrong, or is this a bug in validate_parameters? Thank you
āœ… 1
n

Nate

03/22/2023, 7:41 PM
hi @Mansour Zayer - can you share how you actually defined your flow's signature (just the whole flow definition if possible)?
m

Mansour Zayer

03/22/2023, 7:53 PM
I want to provide the skip arguments from the parallel_orchestrator using **kwargs.
@flow
async def my_subflow(
    input_1: str,
    skip_step_1: bool = False,
    skip_step_2: bool = False,
    skip_step_3: bool = False,
    **kwargs: dict,
    ):
    doing_something

@flow
async def parallel_orchestrator(skip_step: bool = False, list_of_input=["a", "b", "c", "d"], **kwargs:dict[str, bool]):
    some steps here
    await asyncio.gather(
            *[
                run_deployment(
                    name=f"my-subflow/...",
                    parameters=dict(input_1=input, **kwargs),
                )
                for input in list_of_input
            ]
        )
n

Nate

03/22/2023, 8:07 PM
can you show how you're passing in your parameter value? I imagine pydantic is unable to parse what you passed in as a
dict[str, bool]
also just as a side note, I would generally recommend avoiding mutable defaults (like
list_of_input
) for your flow parameters (this is an aggressively worded article šŸ˜… but gets the point across)
šŸ™Œ 1
m

Mansour Zayer

03/22/2023, 8:16 PM
I experimented with different type hints (
dict
,
dict[str, bool]
,
Optional[dict]
), nothing worked.
n

Nate

03/22/2023, 8:32 PM
hmm. i wonder if you actually need to use
**kwargs
here, it feels like it may be simpler and more explicit if you replace it with something like
static_kwargs: Dict[str, Any]
(not using
**
) and just pass each instance of
my_subflow
that dictionary - or is there a reason you can't do that?
šŸ‘€ 1
m

Mansour Zayer

03/22/2023, 8:51 PM
This is the input to the parallel orchestrator which causes failure as invalid dict. Thanks for your help Nate. I'll go with the static_kwargs. I tested it and it achieves my goal.
šŸ‘ 2
n

Nate

03/22/2023, 8:53 PM
ahh I see, sounds good!
z

Zanie

03/22/2023, 9:24 PM
I’m not sure if this is an issue with validation, but when you add a type annotation for
kwargs
you are actually specifying the type for each keyword argument
so
**kwargs: dict
means that you accept any number of keyword arguments that are each a dictionary e.g.
x={}, y={}
šŸ™Œ 1
and
**kwargs: str
means that you accept strings e.g.
x="", y=""
m

Mansour Zayer

03/23/2023, 4:47 PM
Thanks for the good point Zanie. Prefect UI treats
**kwargs
like a regular parameter. For example, it's treated as mandatory, but you can't define default values for
**kwargs
. Also, any type hint you use, the UI will try to cast the
kwargs
value to that type. Therefore, I think this article doesn't apply 100% here.