Running into an interesting issue. We enable data-scientists to define deployments using yaml, and t...
j
Running into an interesting issue. We enable data-scientists to define deployments using yaml, and then during deployment we validate the deployment default arguments against the flow using
Flow.validate_parameters
. This works fine when I use plain function arguments to represent parameters but when I use a Pydantic model (with all of its fields optional) I get an error stating the function argument itself is required. In the example below
validate_parameters
complains that
params
is required.
Copy code
class MyParams(BaseModel):
    refresh_window: str | None = Field(default=None)

@flow
def my_etl_job(params: MyParams):
    ... some work using params
We construct a Flow object from the flow definitions and run
flow.validate_parameters
against the arguments in the yaml configuration. If no configuration is provided we compare it to
{}
.