Hello! I'm running into a weird data validation er...
# prefect-getting-started
y
Hello! I'm running into a weird data validation error, would really appreciate some help. I have a flow that looks like this:
Copy code
@flow
def clean_data_flow(df: pd.DataFrame) -> pd.DataFrame:
    df = remove_nulls(df)
    return df
When I try to run it, I get this error:
File "pydantic/validators.py", line 765, in find_validators
`RuntimeError: no validator found for <class 'pandas.core.frame.DataFrame'>, see
arbitrary_types_allowed
in Config` If I remove the data validators, so my flow looks like this:
Copy code
@flow
def clean_data_flow(df):
    df = remove_nulls(df)
    return df
My code works fine. Not sure why this is happening.
d
Prefect uses pydantic to validate the parameter data types, and pydantic doesn't have a validator for dataframes. If you were creating your own pydantic model you could use
arbitrary_types_allowed
but since you're using
@flow
I don't think you can do that. You can either skip validation as you have done, or pass a more standard type. If you're going to deploy the flow then the parameters also need to be json-serialisable.
y
thanks @Daniel, that was very helpful
1