https://prefect.io logo
Join Slack
Powered by
Hello everyone. I'm having a really weird issue wi...
# ask-community
f
Hello everyone. I'm having a really weird issue with one of my subflows, where it simply crashes with the State message 
Flow run infrastructure exited with non-zero status code 1.
The only log message I can see in the console is 
Downloading flow code from storage at None
. The parent flow works correctly. Are there any server-side logs I can potentially take a look at here? I'm using Prefect 
2.8.5
, but I'll try upgrading to 
>=2.10.7
(couldn't until now because of the pinned 
fsspec
version up 
2.10.7
) to see if that might help. Weirdly, it worked the day before and there have been no changes to the storage blocks...
An upgrade to 
2.10.8
unfortunately did not fix the issue, going to try to debug it a bit further.
After upgrading, I see the following error message in my prefect agent logs:
Copy code
Agent started! Looking for work from work pool 'default-agent-pool'...
/usr/lib/python3.10/runpy.py:126: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
Engine execution of flow run '4ee9705f-d3f6-4db7-8fae-3ea9fe71602b' exited with unexpected exception
Traceback (most recent call last):
  File "/home/prefect/.local/lib/python3.10/site-packages/prefect/engine.py", line 2233, in <module>
    enter_flow_run_engine_from_subprocess(flow_run_id)
  File "/home/prefect/.local/lib/python3.10/site-packages/prefect/engine.py", line 202, in enter_flow_run_engine_from_subprocess
    return from_sync.wait_for_call_in_loop_thread(
  File "/home/prefect/.local/lib/python3.10/site-packages/prefect/_internal/concurrency/api.py", line 215, in wait_for_call_in_loop_thread
    return call.result()
  File "/home/prefect/.local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 173, in result
    return self.future.result(timeout=timeout)
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/home/prefect/.local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 218, in _run_async
    result = await coro
  File "/home/prefect/.local/lib/python3.10/site-packages/prefect/client/utilities.py", line 40, in with_injected_client
    return await fn(*args, **kwargs)
  File "/home/prefect/.local/lib/python3.10/site-packages/prefect/engine.py", line 301, in retrieve_flow_then_begin_flow_run
    if flow.should_validate_parameters:
AttributeError: 'Flow' object has no attribute 'should_validate_parameters'
Process 'congenial-bullfrog' exited with status code: 1
But I'm not quite sure what to make of it. 
flows.py
defines a 
self.should_validate_parameters
attribute at line 
202
, so it seems that there may be some some of version conflict...?
I've verified Both my server and agent are correctly using `2.10.8`: 1. Agent output
Copy code
prefect@prefect:~$ /home/prefect/.local/bin/prefect agent start --pool default-agent-pool --work-queue default
Starting v2.10.8 agent connected to https://<...>/api...
2. Server does not output its version on start, but the /docs page lists the API version as 
2.10.8
so I'm still not sure where the discrepancy comes from.
FWIW: I've created https://discourse.prefect.io/t/flow-run-crashes-with-attributeerror-flow-object-has-no-attribute-should-validate-parameters/2922 for my issue, since I haven't been able to make much progress on this unfortunately.
After lots of debugging I have found the problem to be the schema validation of the 
pendulum.Date
type. I'll try to create a MRE example and create an issue for it.
The root of my issue was being caused by: https://github.com/PrefectHQ/prefect/issues/7502#issuecomment-1545698830. I've updated the discourse thread as well in hopes that it might save someone else a day of debugging. 🙂
Open in Slack
PreviousNext
Powered by