Hey cloud 2 0 user here After upgrading an agent to prefect Prefect Community #ask-community

Hey, cloud 2.0 user here. After upgrading an agent...

Andreas Nigg

04/12/2023, 7:50 AM

Hey, cloud 2.0 user here. After upgrading an agent to prefect 2.10.2 one of my flow runs which runs once per day failed. After manually retriggering the the run, it worked perfectly. From the logs I can't see any issue with the flow itself. The log message which precedes the error was

Copy code

2023-04-12T00:01:35.802260259Z 00:01:35.800 | WARNING | Task run 'get_details_from_db-2' - Task run '17c77b72-df31-4899-836d-23eb934becc5' received abort during orchestration: The enclosing flow must be running to begin task execution. Task run is in PENDING state.

I'm not sure whether this is related to version 2.10.2 - I've just never seen a run failing due to this specific message. The full log output please see in the thread.

Andreas Nigg

04/12/2023, 7:50 AM

Copy code

2023-04-11T23:59:51.631349043Z 23:59:51.628 | INFO    | prefect.agent - Submitting flow run '133f0de9-e1bc-48d4-ba7f-d676039f2c42'
2023-04-11T23:59:52.778922354Z 23:59:52.778 | INFO    | prefect.infrastructure.process - Opening process 'majestic-flamingo'...
2023-04-11T23:59:53.658330459Z 23:59:53.657 | INFO    | prefect.agent - Completed submission of flow run '133f0de9-e1bc-48d4-ba7f-d676039f2c42'
2023-04-11T23:59:55.111233826Z /usr/local/lib/python3.10/runpy.py:126: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
2023-04-11T23:59:55.111260712Z   warn(RuntimeWarning(msg))
2023-04-11T23:59:57.018150496Z 23:59:57.014 | INFO    | Flow run 'majestic-flamingo' - Downloading flow code from storage at None
2023-04-12T00:00:00.155426042Z 00:00:00.153 | INFO    | Flow run 'majestic-flamingo' - Running query select CUSNO,
2023-04-12T00:00:00.155463395Z         ROWTEXT2, ROWTEXT3, COUNTRYCODE, ZIPCODE, CITYNAME, PRODUCT, PACKAGEID, PACKAGENAME, SUBSKIND, SUBSKINDNAME, SUBSNO, EXTNO, SUBSSTATECODE, SUBSSTATENAME, DELSTARTDATE, DELENDDTE, SUBSSTARTDATE, INVSTARTDATE, SUSPENDDATE, CANCELREASONCODE, CANCELREASONNAME, PAYMENTPERIODS, INVOICINGMODE, INVOICINGMODE_DESC, CAMPAIGNID, CAMPAIGNNAME, RECEIVETYPECODE, RECEIVETYPENAME, SUBSQTY, RECOMMENDER, SALESNO, SALESMANNAME, COPRODCODE, COPRODNAME, COPRODVALUE, PAYER, CARRIERNO, BUNDLENO, PRICE, PRICEGROUPCODE, PRICEGROUPNAME, DISCPERCENT, FREESUBSGRPNAME, EMAIL_SSO, SUBSTYPECODE, SUBSTYPE_NAME, SUBSLEN_MONS, SUBSLEN_DAYS, PAPERCODE, PAPERNAME
2023-04-12T00:00:00.155478053Z         from KAYAK.crxvw_longview_subsdata
2023-04-12T00:00:00.155480307Z         where subsstartdate  >= TO_DATE('1800-01-01','yyyy-mm-dd') and subsstartdate < TO_DATE('1990-01-01','yyyy-mm-dd')
2023-04-12T00:00:01.126515144Z 00:00:01.125 | INFO    | Flow run 'majestic-flamingo' - Created task run 'get_details_from_db-0' for task 'get_details_from_db'
2023-04-12T00:00:01.126989983Z 00:00:01.126 | INFO    | Flow run 'majestic-flamingo' - Executing 'get_details_from_db-0' immediately...
2023-04-12T00:00:24.821259632Z 00:00:24.820 | INFO    | prefect.agent - Submitting flow run '133f0de9-e1bc-48d4-ba7f-d676039f2c42'
2023-04-12T00:00:26.317049186Z 00:00:26.316 | INFO    | prefect.infrastructure.process - Opening process 'majestic-flamingo'...
2023-04-12T00:00:26.636413725Z 00:00:26.635 | INFO    | prefect.agent - Completed submission of flow run '133f0de9-e1bc-48d4-ba7f-d676039f2c42'
2023-04-12T00:00:28.637856158Z /usr/local/lib/python3.10/runpy.py:126: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
2023-04-12T00:00:28.637879127Z   warn(RuntimeWarning(msg))
2023-04-12T00:00:29.464202358Z 00:00:29.460 | INFO    | Flow run 'majestic-flamingo' - Downloading flow code from storage at None
2023-04-12T00:00:29.932785986Z 00:00:29.930 | INFO    | Flow run 'majestic-flamingo' - Reading process done
2023-04-12T00:00:32.117605832Z 00:00:32.115 | INFO    | Flow run 'majestic-flamingo' - Running query select CUSNO,
2023-04-12T00:00:32.117641589Z         ROWTEXT2, ROWTEXT3, COUNTRYCODE, ZIPCODE, CITYNAME, PRODUCT, PACKAGEID, PACKAGENAME, SUBSKIND, SUBSKINDNAME, SUBSNO, EXTNO, SUBSSTATECODE, SUBSSTATENAME, DELSTARTDATE, DELENDDTE, SUBSSTARTDATE, INVSTARTDATE, SUSPENDDATE, CANCELREASONCODE, CANCELREASONNAME, PAYMENTPERIODS, INVOICINGMODE, INVOICINGMODE_DESC, CAMPAIGNID, CAMPAIGNNAME, RECEIVETYPECODE, RECEIVETYPENAME, SUBSQTY, RECOMMENDER, SALESNO, SALESMANNAME, COPRODCODE, COPRODNAME, COPRODVALUE, PAYER, CARRIERNO, BUNDLENO, PRICE, PRICEGROUPCODE, PRICEGROUPNAME, DISCPERCENT, FREESUBSGRPNAME, EMAIL_SSO, SUBSTYPECODE, SUBSTYPE_NAME, SUBSLEN_MONS, SUBSLEN_DAYS, PAPERCODE, PAPERNAME
2023-04-12T00:00:32.117645756Z         from KAYAK.crxvw_longview_subsdata
2023-04-12T00:00:32.117647749Z         where subsstartdate  >= TO_DATE('1800-01-01','yyyy-mm-dd') and subsstartdate < TO_DATE('1990-01-01','yyyy-mm-dd')
2023-04-12T00:00:32.447566823Z 00:00:32.446 | INFO    | Flow run 'majestic-flamingo' - Created task run 'get_details_from_db-0' for task 'get_details_from_db'
2023-04-12T00:00:32.447961673Z 00:00:32.447 | INFO    | Flow run 'majestic-flamingo' - Executing 'get_details_from_db-0' immediately...
2023-04-12T00:00:37.586606801Z 00:00:37.584 | INFO    | Flow run 'majestic-flamingo' - Inserting to Bigquery # rows: 131840
2023-04-12T00:00:58.610570366Z 00:00:58.608 | INFO    | Flow run 'majestic-flamingo' - Reading process done
2023-04-12T00:01:01.798023465Z 00:01:01.795 | INFO    | Task run 'get_details_from_db-0' - Finished in state Completed()
2023-04-12T00:01:01.802260432Z 00:01:01.798 | INFO    | Flow run 'majestic-flamingo' - Running query select CUSNO,
2023-04-12T00:01:01.802281076Z         ROWTEXT2, ROWTEXT3, COUNTRYCODE, ZIPCODE, CITYNAME, PRODUCT, PACKAGEID, PACKAGENAME, SUBSKIND, SUBSKINDNAME, SUBSNO, EXTNO, SUBSSTATECODE, SUBSSTATENAME, DELSTARTDATE, DELENDDTE, SUBSSTARTDATE, INVSTARTDATE, SUSPENDDATE, CANCELREASONCODE, CANCELREASONNAME, PAYMENTPERIODS, INVOICINGMODE, INVOICINGMODE_DESC, CAMPAIGNID, CAMPAIGNNAME, RECEIVETYPECODE, RECEIVETYPENAME, SUBSQTY, RECOMMENDER, SALESNO, SALESMANNAME, COPRODCODE, COPRODNAME, COPRODVALUE, PAYER, CARRIERNO, BUNDLENO, PRICE, PRICEGROUPCODE, PRICEGROUPNAME, DISCPERCENT, FREESUBSGRPNAME, EMAIL_SSO, SUBSTYPECODE, SUBSTYPE_NAME, SUBSLEN_MONS, SUBSLEN_DAYS, PAPERCODE, PAPERNAME
2023-04-12T00:01:01.802284696Z         from KAYAK.crxvw_longview_subsdata
2023-04-12T00:01:01.802286557Z         where subsstartdate  >= TO_DATE('1990-01-01','yyyy-mm-dd') and subsstartdate < TO_DATE('2000-01-01','yyyy-mm-dd')
2023-04-12T00:01:02.424262811Z 00:01:02.419 | INFO    | Flow run 'majestic-flamingo' - Created task run 'get_details_from_db-1' for task 'get_details_from_db'
2023-04-12T00:01:02.424836519Z 00:01:02.420 | INFO    | Flow run 'majestic-flamingo' - Executing 'get_details_from_db-1' immediately...
2023-04-12T00:01:06.500101817Z 00:01:06.498 | INFO    | Flow run 'majestic-flamingo' - Inserting to Bigquery # rows: 131840
2023-04-12T00:01:15.996786217Z 00:01:15.995 | INFO    | Flow run 'majestic-flamingo' - Reading process done
2023-04-12T00:01:19.534554662Z 00:01:19.532 | INFO    | Flow run 'majestic-flamingo' - Inserting to Bigquery # rows: 62398
2023-04-12T00:01:27.160389811Z 00:01:27.158 | INFO    | Task run 'get_details_from_db-0' - Task run 'a2ab7e56-a8db-4561-b26e-66bbe0d130a1' already finished.
2023-04-12T00:01:27.164531000Z 00:01:27.160 | ERROR   | Flow run 'majestic-flamingo' - Encountered exception during execution:
2023-04-12T00:01:27.164564604Z Traceback (most recent call last):
2023-04-12T00:01:27.164568327Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 669, in orchestrate_flow_run
2023-04-12T00:01:27.164571707Z     result = await flow_call.aresult()
2023-04-12T00:01:27.164574451Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 181, in aresult
2023-04-12T00:01:27.164577478Z     return await asyncio.wrap_future(self.future)
2023-04-12T00:01:27.164580110Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 194, in _run_sync
2023-04-12T00:01:27.164583444Z     result = self.fn(*self.args, **self.kwargs)
2023-04-12T00:01:27.164586085Z   File "/tmp/tmpakbgej1fprefect/kayak_database_flows.py", line 81, in integrate_kayak_details_full_sync
2023-04-12T00:01:27.164589000Z     get_details_from_db(sql_full_sync, bigquery_dataset_name, bigquery_table_name, timestamp, logger)
2023-04-12T00:01:27.164606105Z   File "/usr/local/lib/python3.10/site-packages/prefect/tasks.py", line 485, in __call__
2023-04-12T00:01:27.164609229Z     return enter_task_run_engine(
2023-04-12T00:01:27.164611965Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 972, in enter_task_run_engine
2023-04-12T00:01:27.164614830Z     return from_sync.wait_for_call_in_loop_thread(begin_run)
2023-04-12T00:01:27.164617490Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/api.py", line 137, in wait_for_call_in_loop_thread
2023-04-12T00:01:27.164620780Z     return call.result()
2023-04-12T00:01:27.164623331Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 173, in result
2023-04-12T00:01:27.164626282Z     return self.future.result(timeout=timeout)
2023-04-12T00:01:27.164628920Z   File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
2023-04-12T00:01:27.164631723Z     return self.__get_result()
2023-04-12T00:01:27.164634241Z   File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
2023-04-12T00:01:27.164637155Z     raise self._exception
2023-04-12T00:01:27.164639724Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 218, in _run_async
2023-04-12T00:01:27.164642747Z     result = await coro
2023-04-12T00:01:27.164645194Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1137, in get_task_call_return_value
2023-04-12T00:01:27.164648170Z     return await future._result()
2023-04-12T00:01:27.164650993Z   File "/usr/local/lib/python3.10/site-packages/prefect/futures.py", line 241, in _result
2023-04-12T00:01:27.164654533Z     return await final_state.result(raise_on_failure=raise_on_failure, fetch=True)
2023-04-12T00:01:27.164657377Z   File "/usr/local/lib/python3.10/site-packages/prefect/states.py", line 98, in _get_state_result
2023-04-12T00:01:27.164660213Z     result = await state.data.get()
2023-04-12T00:01:27.164662740Z   File "/usr/local/lib/python3.10/site-packages/prefect/results.py", line 394, in get
2023-04-12T00:01:27.164665724Z     raise MissingResult("The result was not persisted and is no longer available.")
2023-04-12T00:01:27.164668571Z prefect.exceptions.MissingResult: The result was not persisted and is no longer available.

Andreas Nigg

04/12/2023, 7:51 AM

Copy code

2023-04-12T00:01:27.480033893Z 00:01:27.478 | ERROR   | Flow run 'majestic-flamingo' - Finished in state Failed('Flow run encountered an exception. MissingResult: The result was not persisted and is no longer available.\n')
2023-04-12T00:01:29.674024460Z 00:01:29.591 | ERROR   | prefect.engine - Engine execution of flow run '133f0de9-e1bc-48d4-ba7f-d676039f2c42' exited with unexpected exception
2023-04-12T00:01:29.674048482Z Traceback (most recent call last):
2023-04-12T00:01:29.674051035Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 2158, in <module>
2023-04-12T00:01:29.674060354Z     enter_flow_run_engine_from_subprocess(flow_run_id)
2023-04-12T00:01:29.674062545Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 204, in enter_flow_run_engine_from_subprocess
2023-04-12T00:01:29.674064527Z     ).result()
2023-04-12T00:01:29.674066195Z   File "/usr/local/lib/python3.10/site-packages/prefect/client/schemas.py", line 107, in result
2023-04-12T00:01:29.674068130Z     return get_state_result(self, raise_on_failure=raise_on_failure, fetch=fetch)
2023-04-12T00:01:29.674069957Z   File "/usr/local/lib/python3.10/site-packages/prefect/states.py", line 76, in get_state_result
2023-04-12T00:01:29.674071705Z     return _get_state_result(state, raise_on_failure=raise_on_failure)
2023-04-12T00:01:29.674073399Z   File "/usr/local/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 260, in coroutine_wrapper
2023-04-12T00:01:29.674075421Z     return call()
2023-04-12T00:01:29.674077003Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 245, in __call__
2023-04-12T00:01:29.674078896Z     return self.result()
2023-04-12T00:01:29.674080552Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 173, in result
2023-04-12T00:01:29.674082306Z     return self.future.result(timeout=timeout)
2023-04-12T00:01:29.674084009Z   File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
2023-04-12T00:01:29.674085882Z     return self.__get_result()
2023-04-12T00:01:29.674087505Z   File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
2023-04-12T00:01:29.674089321Z     raise self._exception
2023-04-12T00:01:29.674090913Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 218, in _run_async
2023-04-12T00:01:29.674092820Z     result = await coro
2023-04-12T00:01:29.674094377Z   File "/usr/local/lib/python3.10/site-packages/prefect/states.py", line 91, in _get_state_result
2023-04-12T00:01:29.674096218Z     raise await get_state_exception(state)
2023-04-12T00:01:29.674097866Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 669, in orchestrate_flow_run
2023-04-12T00:01:29.674099657Z     result = await flow_call.aresult()
2023-04-12T00:01:29.674101252Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 181, in aresult
2023-04-12T00:01:29.674103170Z     return await asyncio.wrap_future(self.future)
2023-04-12T00:01:29.674105595Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 194, in _run_sync
2023-04-12T00:01:29.674107471Z     result = self.fn(*self.args, **self.kwargs)
2023-04-12T00:01:29.674109129Z   File "/tmp/tmpakbgej1fprefect/kayak_database_flows.py", line 81, in integrate_kayak_details_full_sync
2023-04-12T00:01:29.674112737Z     get_details_from_db(sql_full_sync, bigquery_dataset_name, bigquery_table_name, timestamp, logger)
2023-04-12T00:01:29.674114631Z   File "/usr/local/lib/python3.10/site-packages/prefect/tasks.py", line 485, in __call__
2023-04-12T00:01:29.674116393Z     return enter_task_run_engine(
2023-04-12T00:01:29.674117991Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 972, in enter_task_run_engine
2023-04-12T00:01:29.674119891Z     return from_sync.wait_for_call_in_loop_thread(begin_run)
2023-04-12T00:01:29.674121611Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/api.py", line 137, in wait_for_call_in_loop_thread
2023-04-12T00:01:29.674123461Z     return call.result()
2023-04-12T00:01:29.674125058Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 173, in result
2023-04-12T00:01:29.674126900Z     return self.future.result(timeout=timeout)
2023-04-12T00:01:29.674128563Z   File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
2023-04-12T00:01:29.674130258Z     return self.__get_result()
2023-04-12T00:01:29.674131859Z   File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
2023-04-12T00:01:29.674133666Z     raise self._exception
2023-04-12T00:01:29.674135220Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 218, in _run_async
2023-04-12T00:01:29.674136992Z     result = await coro
2023-04-12T00:01:29.674138615Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1137, in get_task_call_return_value
2023-04-12T00:01:29.674140421Z     return await future._result()
2023-04-12T00:01:29.674142025Z   File "/usr/local/lib/python3.10/site-packages/prefect/futures.py", line 241, in _result
2023-04-12T00:01:29.674143827Z     return await final_state.result(raise_on_failure=raise_on_failure, fetch=True)
2023-04-12T00:01:29.674145842Z   File "/usr/local/lib/python3.10/site-packages/prefect/states.py", line 98, in _get_state_result
2023-04-12T00:01:29.674147800Z     result = await state.data.get()
2023-04-12T00:01:29.674149466Z   File "/usr/local/lib/python3.10/site-packages/prefect/results.py", line 394, in get
2023-04-12T00:01:29.674151221Z     raise MissingResult("The result was not persisted and is no longer available.")
2023-04-12T00:01:29.674153035Z prefect.exceptions.MissingResult: The result was not persisted and is no longer available.
2023-04-12T00:01:30.178692652Z 00:01:30.177 | ERROR   | prefect.infrastructure.process - Process 'majestic-flamingo' exited with status code: 1
2023-04-12T00:01:35.199720858Z 00:01:35.198 | INFO    | Task run 'get_details_from_db-1' - Finished in state Completed()
2023-04-12T00:01:35.200799681Z 00:01:35.199 | INFO    | Flow run 'majestic-flamingo' - Running query select CUSNO,
2023-04-12T00:01:35.200818079Z         ROWTEXT2, ROWTEXT3, COUNTRYCODE, ZIPCODE, CITYNAME, PRODUCT, PACKAGEID, PACKAGENAME, SUBSKIND, SUBSKINDNAME, SUBSNO, EXTNO, SUBSSTATECODE, SUBSSTATENAME, DELSTARTDATE, DELENDDTE, SUBSSTARTDATE, INVSTARTDATE, SUSPENDDATE, CANCELREASONCODE, CANCELREASONNAME, PAYMENTPERIODS, INVOICINGMODE, INVOICINGMODE_DESC, CAMPAIGNID, CAMPAIGNNAME, RECEIVETYPECODE, RECEIVETYPENAME, SUBSQTY, RECOMMENDER, SALESNO, SALESMANNAME, COPRODCODE, COPRODNAME, COPRODVALUE, PAYER, CARRIERNO, BUNDLENO, PRICE, PRICEGROUPCODE, PRICEGROUPNAME, DISCPERCENT, FREESUBSGRPNAME, EMAIL_SSO, SUBSTYPECODE, SUBSTYPE_NAME, SUBSLEN_MONS, SUBSLEN_DAYS, PAPERCODE, PAPERNAME
2023-04-12T00:01:35.200831169Z         from KAYAK.crxvw_longview_subsdata
2023-04-12T00:01:35.200833348Z         where subsstartdate  >= TO_DATE('2000-01-01','yyyy-mm-dd') and subsstartdate < TO_DATE('2010-01-01','yyyy-mm-dd')
2023-04-12T00:01:35.394623996Z 00:01:35.393 | INFO    | Flow run 'majestic-flamingo' - Created task run 'get_details_from_db-2' for task 'get_details_from_db'
2023-04-12T00:01:35.395026514Z 00:01:35.394 | INFO    | Flow run 'majestic-flamingo' - Executing 'get_details_from_db-2' immediately...
2023-04-12T00:01:35.802260259Z 00:01:35.800 | WARNING | Task run 'get_details_from_db-2' - Task run '17c77b72-df31-4899-836d-23eb934becc5' received abort during orchestration: The enclosing flow must be running to begin task execution. Task run is in PENDING state.
2023-04-12T00:01:35.806021256Z 00:01:35.802 | ERROR   | Flow run 'majestic-flamingo' - Encountered exception during execution:
2023-04-12T00:01:35.806039911Z Traceback (most recent call last):
2023-04-12T00:01:35.806042315Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 669, in orchestrate_flow_run
2023-04-12T00:01:35.806044626Z     result = await flow_call.aresult()
2023-04-12T00:01:35.806046694Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 181, in aresult
2023-04-12T00:01:35.806055567Z     return await asyncio.wrap_future(self.future)
2023-04-12T00:01:35.806057275Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 194, in _run_sync
2023-04-12T00:01:35.806059265Z     result = self.fn(*self.args, **self.kwargs)
2023-04-12T00:01:35.806060885Z   File "/tmp/tmpw2300m5cprefect/kayak_database_flows.py", line 81, in integrate_kayak_details_full_sync
2023-04-12T00:01:35.806062669Z     get_details_from_db(sql_full_sync, bigquery_dataset_name, bigquery_table_name, timestamp, logger)
2023-04-12T00:01:35.806064485Z   File "/usr/local/lib/python3.10/site-packages/prefect/tasks.py", line 485, in __call__
2023-04-12T00:01:35.806066221Z     return enter_task_run_engine(
2023-04-12T00:01:35.806067946Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 972, in enter_task_run_engine
2023-04-12T00:01:35.806069736Z     return from_sync.wait_for_call_in_loop_thread(begin_run)
2023-04-12T00:01:35.806071393Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/api.py", line 137, in wait_for_call_in_loop_thread
2023-04-12T00:01:35.806074343Z     return call.result()
2023-04-12T00:01:35.806077094Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 173, in result
2023-04-12T00:01:35.806088895Z     return self.future.result(timeout=timeout)
2023-04-12T00:01:35.806092272Z   File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
2023-04-12T00:01:35.806095134Z     return self.__get_result()
2023-04-12T00:01:35.806097751Z   File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
2023-04-12T00:01:35.806100545Z     raise self._exception
2023-04-12T00:01:35.806102686Z   File "/usr/local/lib/python3.10/site-packages/prefect/_internal/concurrency/calls.py", line 218, in _run_async
2023-04-12T00:01:35.806105265Z     result = await coro
2023-04-12T00:01:35.806108159Z   File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 1137, in get_task_call_return_value
2023-04-12T00:01:35.806110802Z     return await future._result()
2023-04-12T00:01:35.806113172Z   File "/usr/local/lib/python3.10/site-packages/prefect/futures.py", line 241, in _result
2023-04-12T00:01:35.806116355Z     return await final_state.result(raise_on_failure=raise_on_failure, fetch=True)
2023-04-12T00:01:35.806118957Z   File "/usr/local/lib/python3.10/site-packages/prefect/states.py", line 103, in _get_state_result
2023-04-12T00:01:35.806121530Z     raise MissingResult(
2023-04-12T00:01:35.806124281Z prefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.
2023-04-12T00:01:36.320621638Z 00:01:36.319 | INFO    | prefect.engine - Engine execution of flow run '133f0de9-e1bc-48d4-ba7f-d676039f2c42' aborted by orchestrator: This run has already terminated.
2023-04-12T00:01:39.155375125Z 00:01:39.154 | INFO    | prefect.infrastructure.process - Process 'majestic-flamingo' exited cleanly.

kasteph

05/15/2023, 1:31 PM

I'm also seeing the same error when trying to rerun a Flow:

Copy code

Flow run encountered an exception. MissingResult: The result was not persisted and is no longer available.

I've tried adding state returns explicitly to no avail..

⬆️ 1

Zanie

05/15/2023, 3:24 PM

@kasteph we need the upstream logs, generally that’s an error that occurs because something else went wrong

kasteph

05/15/2023, 3:28 PM

@Zanie yeah that makes sense, because I triggered an exception to reproduce this issue.

2 Views

Open in Slack

Previous Next