David Anderson
06/16/2022, 9:02 PMAirbyteConnectionTask
seeing a new error starting ~last week? all of my runs are failing with Error during execution of task: KeyError('schedule')
. haven't changed a thing in my prefect flow configuration. thought maybe it was related to an airbyte upgrade (im running the self-hosted version), but im struggling to troubleshoot. any ideas?Kevin Kho
06/16/2022, 10:53 PMDavid Anderson
06/16/2022, 10:59 PMTraceback (most recent call last):
File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 884, in get_task_run_state
logger=self.logger,
File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 456, in method
return run_method(self, *args, **kwargs)
File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 341, in run
session, airbyte_base_url, connection_id
File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 181, in _get_connection_status
schedule = response.json()["schedule"]
KeyError: 'schedule'
Kevin Kho
06/16/2022, 11:07 PMDavid Anderson
06/16/2022, 11:08 PMKevin Kho
06/16/2022, 11:14 PMschedule
in the response. The only scenario I can see this happening in is if you get a non-200 status code back. We could add a guard against that, but for you, is this the first time you used it or has it worked before? Because I think there’s a chance you gave an object id that may not exist?David Anderson
06/17/2022, 1:45 PMKevin Kho
06/17/2022, 2:00 PMDavid Anderson
06/21/2022, 1:36 PMAttributeError("'AirbyteConnectionTask' object has no attribute 'stream_output'")
. the relevant section of the log is below.
im guessing the issue is as you suspected -- something changing with the airbyte api. the new error started presenting after i updated airbyte to 0.39.31-alpha (the newest release as-of yesterday).
Task 'AirbyteConnectionTask': Exception encountered during task execution!
Traceback (most recent call last):
File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 884, in get_task_run_state
logger=self.logger,
File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 456, in method
return run_method(self, *args, **kwargs)
File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 341, in run
session, airbyte_base_url, connection_id
File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 176, in _get_connection_status
self.logger.log(level=self.stream_output, msg=response.json())
AttributeError: 'AirbyteConnectionTask' object has no attribute 'stream_output'
Kevin Kho
06/21/2022, 1:45 PMDavid Anderson
06/21/2022, 2:05 PMKevin Kho
06/21/2022, 4:03 PMDavid Anderson
06/21/2022, 4:22 PM