Soham
07/12/2023, 11:38 PMNate
07/12/2023, 11:39 PMMarvin
07/12/2023, 11:39 PMNate
07/12/2023, 11:39 PMSoham
07/12/2023, 11:48 PMmaximum recursion depth exceeded in comparison
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/prefect/flows.py", line 511, in __call__
return enter_flow_run_engine_from_flow_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/prefect/engine.py", line 208, in enter_flow_run_engine_from_flow_call
retval = from_sync.wait_for_call_in_loop_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/prefect/_internal/concurrency/api.py", line 243, in wait_for_call_in_loop_thread
return call.result()
^^^^^^^^^^^^^
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 283, in result
return self.future.result(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 169, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 346, in _run_async
result = await coro
^^^^^^^^^^
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/prefect/client/utilities.py", line 40, in with_injected_client
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/prefect/engine.py", line 563, in create_and_begin_subflow_run
parameters=flow.serialize_parameters(parameters),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/prefect/flows.py", line 415, in serialize_parameters
serialized_parameters[key] = jsonable_encoder(value)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/fastapi/encoders.py", line 131, in jsonable_encoder
jsonable_encoder(
File "/Users/soham/anaconda3/envs/echoflow/lib/python3.11/site-packages/fastapi/encoders.py", line 66, in jsonable_encoder
return jsonable_encoder(
Marvin
07/13/2023, 12:14 AM~/.prefect/prefect.db
. They suggested reducing the number of logs to avoid overloading the database and setting concurrency limits on some tasks[^1^].
Without seeing your code, it's hard to say exactly what's causing the error. I would suggest checking your flow logic to make sure there's a condition that stops the creation of new nested flows at some point. You might also want to consider whether there's a more efficient way to structure your flows that doesn't require such deep nesting.
And remember, just because you can nest a flow within a flow within a flow, doesn't always mean you should. But then again, who am I to stand in the way of human curiosity?
[^1^]: (https://discourse.prefect.io/t/scheduled-flow-every-30min-crashed-after-running-on-local-server-for-2-days/2816)
</stop>Soham
07/13/2023, 12:40 AMMarvin
07/13/2023, 12:41 AM@flow
decorator. It's the basis of all Prefect workflows and must contain at least one flow function that serves as the entry point for execution. Flows can include calls to tasks as well as to child flows, which are referred to as "subflows" in this context[^1^].
A task, on the other hand, is a Python function decorated with a @task
decorator. Tasks represent distinct pieces of work executed within a flow. All tasks must be called from within a flow and may not be called from other tasks[^2^].
When deciding whether to use a subflow or a task, consider the nature of the work you're doing. Tasks should do a discrete, specific piece of work in your workflow, like calling an API or performing a database operation. Subflows are useful for grouping related tasks within your workflow[^3^].
So, in your case, if your function is doing a specific piece of work and doesn't have any dependencies, it might be more appropriate to define it as a task rather than a flow.
Remember, just because you can nest a flow within a flow within a flow, doesn't always mean you should. But then again, who am I to stand in the way of human ambition?
[^1^]: (https://docs.prefect.io/latest/tutorial/first-steps/)
[^2^]: (https://docs.prefect.io/latest/concepts/tasks/)
[^3^]: (https://docs.prefect.io/latest/concepts/flows/)
</stop>Nate
07/13/2023, 1:17 AMSoham
07/13/2023, 2:55 AM