https://prefect.io logo
Title
d

David Anderson

12/07/2022, 5:09 PM
i recently upgraded our airbyte from 40.14 to 40.23, and im now encountering an error with prefect (1.2.4) triggering airbyte runs. the error message we're seeing in our logs is below. anyone else run into a similar issue with newer releases of airbyte recently? im wondering / guessing if this is related to a change in the airbyte API.
Task 'AirbyteConnectionTask': Exception encountered during task execution!
Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.7/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib64/python3.7/site-packages/simplejson/__init__.py", line 488, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.7/site-packages/simplejson/decoder.py", line 374, in decode
    obj, end = self.raw_decode(s)
  File "/usr/lib64/python3.7/site-packages/simplejson/decoder.py", line 393, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 67, in _check_health_status
    self.logger.debug("Health check response: %s", response.json())
  File "/home/ec2-user/.local/lib/python3.7/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 884, in get_task_run_state
    logger=self.logger,
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 456, in method
    return run_method(self, *args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 333, in run
    session = airbyte._establish_session()
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 60, in _establish_session
    if self._check_health_status(session):
  File "/home/ec2-user/.local/lib/python3.7/site-packages/prefect/tasks/airbyte/airbyte.py", line 76, in _check_health_status
    raise AirbyteServerNotHealthyException(e)
prefect.tasks.airbyte.airbyte.AirbyteServerNotHealthyException: Expecting value: line 1 column 1 (char 0)
figured it out after digging around in the airbyte slack. this is in fact related to the basic auth / password that airbyte implemented. if you remove basic auth in the .env, this issue gets resolved.
n

Nate

12/07/2022, 5:52 PM
Hey @David Anderson just to note, I saw you're on prefect 1.2.4 but if / when you switch to prefect 2.x, we've got support for Airbyte's new basic auth here!
d

David Anderson

12/07/2022, 6:21 PM
thanks, @Nate .. any chance you all have a migration guide to upgrade to 2.0? i seem to remember its not lift-and-drop .. but may be a good weekend project 🙂
n

Nate

12/07/2022, 6:23 PM
Yep you're right, there are some significant changes in the way we define / deploy flows and store configuration. Here's a general rundown, but feel free to ask specific questions here!