Hello! I'm running into a weird data validation error, would really appreciate some help. I have a flow that looks like this:

@flow
def clean_data_flow(df: pd.DataFrame) -> pd.DataFrame:
    df = remove_nulls(df)
    return df

When I try to run it, I get this error:

File "pydantic/validators.py", line 765, in find_validators

`RuntimeError: no validator found for <class 'pandas.core.frame.DataFrame'>, see

arbitrary_types_allowed

in Config` If I remove the data validators, so my flow looks like this:

@flow
def clean_data_flow(df):
    df = remove_nulls(df)
    return df

My code works fine. Not sure why this is happening.

Prefect uses pydantic to validate the parameter data types, and pydantic doesn't have a validator for dataframes. If you were creating your own pydantic model you could use

arbitrary_types_allowed

but since you're using

@flow

I don't think you can do that. You can either skip validation as you have done, or pass a more standard type. If you're going to deploy the flow then the parameters also need to be json-serialisable.

thanks @Daniel, that was very helpful