Prefect Error When Subflow Starts Execution Hey All I m enc Prefect Community #ask-community

Prefect Error When Subflow Starts Execution Hey A...

ravi

10/20/2023, 10:52 AM

Prefect Error When Subflow Starts Execution Hey All, I’m encountering an error with Prefect when my subflow begins its execution. I suspect there’s a minor detail I’m overlooking. Has anyone come across a similar issue before? I’m happy to provide more context or share error logs if needed. Thanks in advance!

Copy code

File "/venv/lib/python3.10/site-packages/prefect/states.py", line 91, in _get_state_result
    raise await get_state_exception(state)
  File "/venv/lib/python3.10/site-packages/prefect/engine.py", line 647, in create_and_begin_subflow_run
    parameters = flow.validate_parameters(parameters)
  File "/venv/lib/python3.10/site-packages/prefect/flows.py", line 479, in validate_parameters
    model = validated_fn.init_model_instance(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/pydantic/v1/decorator.py", line 130, in init_model_instance
    return self.model(**values)
  File "/venv/lib/python3.10/site-packages/pydantic/main.py", line 164, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
TypeError: _validate_dataclass() takes 2 positional arguments but 3 were given
08:09:52 PM
prefect.flow_runs
Finished in state Failed('Flow run encountered an exception. TypeError: _validate_dataclass() takes 2 positional arguments but 3 were given')

Brandon Robertson

10/23/2023, 8:06 PM

Hi Rav, this exact error started to occur in our production pipeline a few hours ago. Any resolution?

Brandon Robertson

10/23/2023, 8:11 PM

Here's my stack trace:

Copy code

File "/usr/local/lib/python3.10/site-packages/prefect/flows.py", line 1045, in __call__
    return enter_flow_run_engine_from_flow_call(
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 271, in enter_flow_run_engine_from_flow_call
    retval = from_sync.wait_for_call_in_loop_thread(
  File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/api.py", line 243, in wait_for_call_in_loop_thread
    return call.result()
  File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 282, in result
    return self.future.result(timeout=timeout)
  File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 168, in result
    return self.__get_result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 345, in _run_async
    result = await coro
  File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 713, in create_and_begin_subflow_run
    return await terminal_state.result(fetch=True)
  File "/usr/local/lib/python3.10/site-packages/prefect/states.py", line 91, in _get_state_result
    raise await get_state_exception(state)
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 647, in create_and_begin_subflow_run
    parameters = flow.validate_parameters(parameters)
  File "/usr/local/lib/python3.10/site-packages/prefect/flows.py", line 479, in validate_parameters
    model = validated_fn.init_model_instance(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pydantic/v1/decorator.py", line 130, in init_model_instance
    return self.model(**values)
  File "/usr/local/lib/python3.10/site-packages/pydantic/main.py", line 164, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
TypeError: _validate_dataclass() takes 2 positional arguments but 3 were given

Brandon Robertson

10/23/2023, 8:48 PM

Setting

validate_parameters=False

inside flow decorator bypassed the pydantic validation and subflow now running successfully. Still worrisome that this issue showed up randomly without any changes to code/prefect deployment.

ravi

10/24/2023, 12:57 PM

Hey @Brandon Robertson yep, agree this is worrisome Removing all type hints from subflow function signatures fixes the error for us. Our guess is it’s something to do with how Prefect handles parameters with subflows, possibly confusing our typehint for a Pydantic Model? cc: @alex for your attention

alex

10/24/2023, 1:50 PM

We removed our upper pin on

pydantic

recently, so this behavior may manifest because you had

pydantic

v1 installed before and it changed to

pydantic

v2 on an upgrade of the

prefect

version. If you open an issue with details of your

prefect

and

pydantic

we can investigate further.

16 Views

Open in Slack

Previous Next