is anyone who uses

AirbyteConnectionTask

seeing a new error starting ~last week? all of my runs are failing with

Error during execution of task: KeyError('schedule')

. haven't changed a thing in my prefect flow configuration. thought maybe it was related to an airbyte upgrade (im running the self-hosted version), but im struggling to troubleshoot. any ideas?

Hey David, can I see the traceback you get?

sure thing @Kevin Kho

Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 884, in get_task_run_state
    logger=self.logger,
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 456, in method
    return run_method(self, *args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 341, in run
    session, airbyte_base_url, connection_id
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 181, in _get_connection_status
    schedule = response.json()["schedule"]
KeyError: 'schedule'

What is your Prefect version?

1.2.2

Ok so the code looks right. Your error is from this line which hits this endpoint, which does contain a

schedule

in the response. The only scenario I can see this happening in is if you get a non-200 status code back. We could add a guard against that, but for you, is this the first time you used it or has it worked before? Because I think there’s a chance you gave an object id that may not exist?

@Kevin Kho these three scheduled flows were running fine for a few weeks before all failing about a week back.

Gotcha I guess they have updated their API. Will look into it

hey @Kevin Kho, wanted to share a new piece of info on this. im now getting a new error: Error during execution of task:

AttributeError("'AirbyteConnectionTask' object has no attribute 'stream_output'")

. the relevant section of the log is below. im guessing the issue is as you suspected -- something changing with the airbyte api. the new error started presenting after i updated airbyte to 0.39.31-alpha (the newest release as-of yesterday).

Task 'AirbyteConnectionTask': Exception encountered during task execution!
Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 884, in get_task_run_state
    logger=self.logger,
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 456, in method
    return run_method(self, *args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 341, in run
    session, airbyte_base_url, connection_id
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 176, in _get_connection_status
    self.logger.log(level=self.stream_output, msg=response.json())
AttributeError: 'AirbyteConnectionTask' object has no attribute 'stream_output'

Hey David, thanks for following up. The people with Airbbyte knowledge have just gotten back from vacation so I’ll forward this

🙏 much appreciated!

So looks like there was a fix for the schedule one here: https://github.com/PrefectHQ/prefect/commit/7ce5d1ad25f57bfbbc31f69101693a9c3a74073f

awesome, thanks for the heads-up