Hi, is there any insights on prefect having old flow runs using pydantic V1 data models still retrievable via client with prefect 2.18+ with pydantic v2 in place?

Hey Duy! Can you say more here — are you encountering an error?

Yes. I’m trying to upgrade data models in my project to pydantic 2.7. After the upgrade, I’m seeing that I’d need to upgrade prefect to at least 2.18 as pydantic 2 is supported there. Then, I tried to run prefect client with 2.18 and pydantic 2.7 to fetch historical data and could not get it due to

no validator found for <class 'this.is.my.DataClass'>, see `arbitrary_types_allowed` in Config

Gotcha. This isn’t quite enough for me to reproduce — if you wanted to share more code (like what the

DataClass

model is and how you’re using it to postprocess results (?)) typically I see that

no validator

error when I try to use a v1 basemodel as a type on a v2 basemodel.

I think you already point to the issue. So, if we have

DataClass

was defined with v1 yesterday and the flow run result of type

DataClass

was persisted to S3 with prefect server v2.16. Then, today, we upgrade prefect server to v2.18 and having prefect client v2.18 and we pull the flow run data with the script I mentioned above. The error will be thrown with

no validator.

.

I believe so - yep!

@Alexander Azzam Here’s my traceback

Traceback (most recent call last):
  File "src/app/flow/get_flow_run.py", line 61, in get_flow_run
    await flow_run.state.result(fetch=True)  # type: ignore[call-overload]
  File ".venv/lib/python3.10/site-packages/prefect/states.py", line 98, in _get_state_result
    result = await state.data.get()
  File ".venv/lib/python3.10/site-packages/prefect/client/utilities.py", line 100, in with_injected_client
    return await fn(*args, **kwargs)
  File ".venv/lib/python3.10/site-packages/prefect/results.py", line 605, in get
    obj = blob.serializer.loads(blob.data)
  File ".venv/lib/python3.10/site-packages/prefect/serializers.py", line 220, in loads
    return json.loads(blob.decode(), **kwargs)
  File "/.pyenv/versions/3.10.11/lib/python3.10/json/__init__.py", line 359, in loads
    return cls(**kw).decode(s)
  File "/.pyenv/versions/3.10.11/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/.pyenv/versions/3.10.11/lib/python3.10/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
  File ".venv/lib/python3.10/site-packages/prefect/serializers.py", line 84, in prefect_json_object_decoder
    return parse_obj_as(from_qualified_name(result["__class__"]), result["data"])
  File ".venv/lib/python3.10/site-packages/pydantic/v1/tools.py", line 37, in parse_obj_as
    model_type = _get_parsing_type(type_, type_name=type_name)  # type: ignore[arg-type]
  File ".venv/lib/python3.10/site-packages/pydantic/v1/tools.py", line 30, in _get_parsing_type
    return create_model(type_name, __root__=(type_, ...))
  File ".venv/lib/python3.10/site-packages/pydantic/v1/main.py", line 1024, in create_model
    return meta(__model_name, resolved_bases, namespace, **kwds)
  File ".venv/lib/python3.10/site-packages/pydantic/v1/main.py", line 197, in __new__
    fields[ann_name] = ModelField.infer(
  File ".venv/lib/python3.10/site-packages/pydantic/v1/fields.py", line 504, in infer
    return cls(
  File ".venv/lib/python3.10/site-packages/pydantic/v1/fields.py", line 434, in __init__
    self.prepare()
  File ".venv/lib/python3.10/site-packages/pydantic/v1/fields.py", line 555, in prepare
    self.populate_validators()
  File ".venv/lib/python3.10/site-packages/pydantic/v1/fields.py", line 829, in populate_validators
    *(get_validators() if get_validators else list(find_validators(self.type_, self.model_config))),
  File ".venv/lib/python3.10/site-packages/pydantic/v1/validators.py", line 765, in find_validators
    raise RuntimeError(f'no validator found for {type_}, see `arbitrary_types_allowed` in Config')
RuntimeError: no validator found for <class 'app.flow.flow_response_model.FlowResult'>, see `arbitrary_types_allowed` in Config

Can I see the FlowResult class with its imports?

class FlowResult(BaseModel):
    """
    Pydantic model for the result of the app flow.

    Args:
        task_1 (Optional[Task1Result]): The result task_1 produced.
        task_2 (Optional[Task2Result]): The result task_2 produced.
    """

    task_1: Optional[Task1Result] = Field(default=None)
    task_2: Optional[Task1Result] = Field(default=None)

Here’s the content of it.

Thanks!

@Alexander Azzam were you able to reproduce? I think I got the point of failure in

prefect/serializers.py

As

Serializer

there is still using V1 BaseModel and

FlowResult

is using V2 BaseModel. It’s not possible to use

parse_obj_as

of

pydantic.v1

but should use

parse_obj_as

in

pydantic

and everything will works.