<@ULVA73B9P> : Why does this timeout when I haven'...
# ask-marvin
s
@Marvin : Why does this timeout when I haven't set any timeout to a flow or task explicitly?
Copy code
File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/distributed/client.py", line 336, in _result
    raise exc.with_traceback(tb)
  File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/prefect/client/base.py", line 129, in app_lifespan_context
    await context.__aenter__()
  File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/asgi_lifespan/_manager.py", line 99, in __aenter__
    await self.startup()
  File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/asgi_lifespan/_manager.py", line 40, in startup
    await self._concurrency_backend.run_and_fail_after(
  File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/asgi_lifespan/_concurrency/asyncio.py", line 46, in run_and_fail_after
    raise TimeoutError
TimeoutError
01:38:30 AM
prefect.flow_runs
Finished in state Failed('Flow run encountered an exception. TimeoutError: ')
m
Your message was too long by 625 tokens - please shorten it and try again. For reference, here's your message at the allowed limit:
:
Why does this timeout when I haven't set any timeout to a flow or task explicitly? ``` File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/echodataflow/stages/subflows/open_raw.py", line 94, in ed_list = [f.result() for f in futures] File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/prefect/futures.py", line 228, in result return from_sync.call_soon_in_loop_thread(result).result() File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 318, in result return self.future.result(timeout=timeout) File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 188, in result return self.__get_result() File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 389, in _run_async result = await coro
Your message was too long by 196 tokens - please shorten it and try again. For reference, here's your message at the allowed limit:
:
Why does this timeout when I haven't set any timeout to a flow or task explicitly? ``` File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/distributed/client.py", line 336, in _result raise exc.with_traceback(tb) File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/prefect/engine.py", line 1791, in begin_task_run client = await stack.enter_async_context(get_client()) File "/usr/lib/python3.10/contextlib.py", line 619, in enter_async_context result = await _cm_type.__aenter__(cm) File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/prefect/client/orchestration.py", line 3323, in aenter self._ephemeral_lifespan = await self._exit_stack.enter_async_context( File "/usr/lib/python3.10/contextlib.py", line 619, in enter_async_context result = await _cm_type.__aenter__(cm) File "/usr/lib/python3.10/contextlib.py", line 199, in aenter return await anext(self.gen) File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/prefect/client/base.py
Your message was too long by 36 tokens - please shorten it and try again. For reference, here's your message at the allowed limit:
:
Why does this timeout when I haven't set any timeout to a flow or task explicitly? ``` File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/distributed/client.py", line 336, in _result raise exc.with_traceback(tb) File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/prefect/engine.py", line 1791, in begin_task_run client = await stack.enter_async_context(get_client()) File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/prefect/client/base.py", line 129, in app_lifespan_context await context.__aenter__() File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/asgi_lifespan/_manager.py", line 99, in aenter await self.startup() File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/asgi_lifespan/_manager.py", line 40, in startup await self._concurrency_backend.run_and_fail_after( File "/home/exouser/Desktop/Echodataflow/echodataflow/lib/python3.10/site-packages/asgi_lifespan/_concurrency/asyncio.py", line 46, in run_and_fail_after
It appears you are encountering a
TimeoutError
during a flow run, but you haven't explicitly set any timeouts. Based on the traceback, the timeout seems to be related to Dask's asynchronous client behavior or to Prefect internals when dealing with suspended or paused flow runs, rather than a flow or task-level timeout set by you. In Prefect, there are default timeouts for suspended runs. Additionally, there are GitHub issues that describe similar problems, which suggest that the error could be related to nested blocks leading to a
TimeoutError
when loaded in a task (issue #8976), or caused by missing state data when the worker's memory overflows (issue #12977). To resolve this, you might need to look into proper configuration for using Dask with Prefect, making sure your task execution does not hit any underlying resource limits, or inspect block-loading patterns that could be causing nested blocks to load incorrectly. For using Dask with Prefect, you can refer to Prefect’s documentation: - Dask integration guide - Concepts of Flows, timeouts, pausing, and suspending For the nested block issue detailed in GitHub issue #8976 and the missing result state problem as in GitHub issue #12977, it's recommended to follow the progress on these issues and apply any proposed solutions or workarounds discussed there. Unfortunately, my capabilities don't include analyzing or debugging application-specific issues in-depth, but I hope pointing you to these resources and discussions helps nudge you in the right direction.