Jonas Hanfland
10/06/2020, 4:43 PMBigQueryTask
.
This is the task I am trying to run:
BigQueryTask(
name="Get all statement IDs",
query="""
SELECT * FROM temporary_tables.statements
""",
max_retries=3,
retry_delay=timedelta(seconds=30),
to_dataframe=True,
)
and it throws this exception:
(see thread)
All the times this task has been run in the past it passed just fine.
Does anybody have an idea what might be the issue? Thanks very much in advance!Chris White
Jonas Hanfland
10/06/2020, 4:48 PM[2020-10-06 15:39:25] ERROR - prefect.TaskRunner | Unexpected error: AttributeError("'BigQueryReadClient' object has no attribute 'transport'")
Traceback (most recent call last):
File "/home/venv/lib/python3.8/site-packages/google/cloud/bigquery/table.py", line 1545, in to_arrow
for record_batch in self._to_arrow_iterable(
File "/home/venv/lib/python3.8/site-packages/google/cloud/bigquery/table.py", line 1435, in _to_page_iterable
for item in bqstorage_download():
File "/home/venv/lib/python3.8/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 629, in _download_table_bqstorage
from google.cloud import bigquery_storage_v1beta1
ImportError: cannot import name 'bigquery_storage_v1beta1' from 'google.cloud' (unknown location)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/venv/lib/python3.8/site-packages/prefect/engine/runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "/home/venv/lib/python3.8/site-packages/prefect/engine/task_runner.py", line 824, in get_task_run_state
value = timeout_handler(
File "/home/venv/lib/python3.8/site-packages/prefect/utilities/executors.py", line 188, in timeout_handler
return fn(*args, **kwargs)
File "/home/venv/lib/python3.8/site-packages/prefect/utilities/tasks.py", line 449, in method
return run_method(self, *args, **kwargs)
File "/home/venv/lib/python3.8/site-packages/prefect/tasks/gcp/bigquery.py", line 181, in run
return query_job.result().to_dataframe()
File "/home/venv/lib/python3.8/site-packages/google/cloud/bigquery/table.py", line 1725, in to_dataframe
record_batch = self.to_arrow(
File "/home/venv/lib/python3.8/site-packages/google/cloud/bigquery/table.py", line 1562, in to_arrow
bqstorage_client.transport.channel.close()
AttributeError: 'BigQueryReadClient' object has no attribute 'transport'
Name: google-cloud-bigquery-storage
Version: 2.0.0
Chris White
Jonas Hanfland
10/06/2020, 4:56 PMChris White