https://prefect.io logo
#prefect-community
Title
# prefect-community
d

David Anderson

06/16/2022, 9:02 PM
is anyone who uses
AirbyteConnectionTask
seeing a new error starting ~last week? all of my runs are failing with
Error during execution of task: KeyError('schedule')
. haven't changed a thing in my prefect flow configuration. thought maybe it was related to an airbyte upgrade (im running the self-hosted version), but im struggling to troubleshoot. any ideas?
k

Kevin Kho

06/16/2022, 10:53 PM
Hey David, can I see the traceback you get?
d

David Anderson

06/16/2022, 10:59 PM
sure thing @Kevin Kho
Copy code
Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 884, in get_task_run_state
    logger=self.logger,
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 456, in method
    return run_method(self, *args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 341, in run
    session, airbyte_base_url, connection_id
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 181, in _get_connection_status
    schedule = response.json()["schedule"]
KeyError: 'schedule'
k

Kevin Kho

06/16/2022, 11:07 PM
What is your Prefect version?
d

David Anderson

06/16/2022, 11:08 PM
1.2.2
k

Kevin Kho

06/16/2022, 11:14 PM
Ok so the code looks right. Your error is from this line which hits this endpoint, which does contain a
schedule
in the response. The only scenario I can see this happening in is if you get a non-200 status code back. We could add a guard against that, but for you, is this the first time you used it or has it worked before? Because I think there’s a chance you gave an object id that may not exist?
d

David Anderson

06/17/2022, 1:45 PM
@Kevin Kho these three scheduled flows were running fine for a few weeks before all failing about a week back.
k

Kevin Kho

06/17/2022, 2:00 PM
Gotcha I guess they have updated their API. Will look into it
🙌 1
d

David Anderson

06/21/2022, 1:36 PM
hey @Kevin Kho, wanted to share a new piece of info on this. im now getting a new error: Error during execution of task:
AttributeError("'AirbyteConnectionTask' object has no attribute 'stream_output'")
. the relevant section of the log is below. im guessing the issue is as you suspected -- something changing with the airbyte api. the new error started presenting after i updated airbyte to 0.39.31-alpha (the newest release as-of yesterday).
Copy code
Task 'AirbyteConnectionTask': Exception encountered during task execution!
Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 884, in get_task_run_state
    logger=self.logger,
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 456, in method
    return run_method(self, *args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 341, in run
    session, airbyte_base_url, connection_id
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 176, in _get_connection_status
    self.logger.log(level=self.stream_output, msg=response.json())
AttributeError: 'AirbyteConnectionTask' object has no attribute 'stream_output'
k

Kevin Kho

06/21/2022, 1:45 PM
Hey David, thanks for following up. The people with Airbbyte knowledge have just gotten back from vacation so I’ll forward this
d

David Anderson

06/21/2022, 2:05 PM
🙏 much appreciated!
k

Kevin Kho

06/21/2022, 4:03 PM
So looks like there was a fix for the schedule one here: https://github.com/PrefectHQ/prefect/commit/7ce5d1ad25f57bfbbc31f69101693a9c3a74073f
will be out next release
d

David Anderson

06/21/2022, 4:22 PM
awesome, thanks for the heads-up
10 Views