https://prefect.io logo
Title
m

Mark NS

05/05/2023, 6:49 AM
Hi, I'm getting a
RecursionError: maximum recursion depth exceeded in comparison
very similar to this (unanswered) report from @Austin Weisgrau a few days ago. Full stack trace in thread. Could someone from the Prefect team take a look? (also, we're paying customers, not sure if that entitles us to raise support tickets beyond here and/or github somehow?)
Version:             2.8.4
API version:         0.8.4
Python version:      3.10.3
Git commit:          f09ccc32
Built:               Thu, Mar 2, 2023 12:34 PM
OS/Arch:             darwin/arm64
Profile:             dev
Server type:         cloud
Encountered exception during execution:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 654, in orchestrate_flow_run
    result = await run_sync(flow_call)
  File "/usr/local/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 154, in run_sync_in_interruptible_worker_thread
    async with anyio.create_task_group() as tg:
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
    raise exceptions[0]
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 135, in capture_worker_thread_and_result
    result = __fn(*args, **kwargs)
  File "/work/flows/transformation/ditto/dbt_build.py", line 26, in ditto
    dbt.dbt_cli("dbt debug", task_options)
  File "/work/alakazam/dbt.py", line 70, in dbt_cli
    return state.result(raise_on_failure=stop_on_failure)
  File "/usr/local/lib/python3.10/site-packages/prefect/client/schemas.py", line 107, in result
    return get_state_result(self, raise_on_failure=raise_on_failure, fetch=fetch)
  File "/usr/local/lib/python3.10/site-packages/prefect/states.py", line 76, in get_state_result
    return _get_state_result(state, raise_on_failure=raise_on_failure)
  File "/usr/local/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 226, in coroutine_wrapper
    return run_async_from_worker_thread(async_fn, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 177, in run_async_from_worker_thread
    return anyio.from_thread.run(call)
  File "/usr/local/lib/python3.10/site-packages/anyio/from_thread.py", line 49, in run
    return asynclib.run_async_from_thread(func, *args)
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 970, in run_async_from_thread
    return f.result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.10/site-packages/prefect/states.py", line 91, in _get_state_result
    raise await get_state_exception(state)
  File "/usr/local/lib/python3.10/site-packages/prefect/task_runners.py", line 207, in submit
    result = await call()
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1340, in begin_task_run
    state = await orchestrate_task_run(
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1460, in orchestrate_task_run
    state = await propose_state(
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1879, in propose_state
    return await propose_state(
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1879, in propose_state
    return await propose_state(
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1879, in propose_state
    return await propose_state(
  [Previous line repeated 945 more times]
  File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1846, in propose_state
    response = await client.set_task_run_state(
  File "/usr/local/lib/python3.10/site-packages/prefect/client/orchestration.py", line 1923, in set_task_run_state
    return OrchestrationResult.parse_obj(response.json())
  File "pydantic/main.py", line 527, in pydantic.main.BaseModel.parse_obj
  File "pydantic/main.py", line 340, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1077, in pydantic.main.validate_model
  File "pydantic/fields.py", line 884, in pydantic.fields.ModelField.validate
  File "pydantic/fields.py", line 1101, in pydantic.fields.ModelField._validate_singleton
  File "pydantic/fields.py", line 1148, in pydantic.fields.ModelField._apply_validators
  File "pydantic/class_validators.py", line 318, in pydantic.class_validators._generic_validator_basic.lambda13
  File "pydantic/main.py", line 712, in pydantic.main.BaseModel.validate
  File "pydantic/main.py", line 340, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1077, in pydantic.main.validate_model
  File "pydantic/fields.py", line 884, in pydantic.fields.ModelField.validate
  File "pydantic/fields.py", line 1101, in pydantic.fields.ModelField._validate_singleton
  File "pydantic/fields.py", line 1148, in pydantic.fields.ModelField._apply_validators
  File "pydantic/class_validators.py", line 318, in pydantic.class_validators._generic_validator_basic.lambda13
  File "pydantic/main.py", line 712, in pydantic.main.BaseModel.validate
  File "pydantic/main.py", line 340, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1077, in pydantic.main.validate_model
  File "pydantic/fields.py", line 884, in pydantic.fields.ModelField.validate
  File "pydantic/fields.py", line 1101, in pydantic.fields.ModelField._validate_singleton
  File "pydantic/fields.py", line 1148, in pydantic.fields.ModelField._apply_validators
  File "pydantic/class_validators.py", line 322, in pydantic.class_validators._generic_validator_basic.lambda15
  File "pydantic/validators.py", line 318, in pydantic.validators.uuid_validator
  File "/usr/local/lib/python3.10/uuid.py", line 170, in __init__
    if [hex, bytes, bytes_le, fields, int].count(None) != 4:
RecursionError: maximum recursion depth exceeded in comparison
m

Mason Menges

05/05/2023, 2:15 PM
Hey @Mark NS I believe this might be related to an issue that was fixed with the 2.8.7 release Fix bug where propose_state could exceed recursion limits during extended waits — #8827 Are you able to retest this after updating the Prefect Version to 2.8.7 or greater?
m

Mark NS

05/05/2023, 2:16 PM
Thanks @Mason Menges, I'll test with the latest version of Prefect next week.
m

Mason Menges

05/05/2023, 2:17 PM
Brilliant 😄 let us know if you still run into issues with it after that.