https://prefect.io logo
Channels
Powered by
c

Charles Leung

01/14/2021, 3:08 PM
Hey prefect team, just wanted to let you know a heads up on installing GCP extras with pip - 🙂 i ran into this issue where it kept saying it needed to be installed as a prefect extra, but realized after i removed the try/except in the _init that_ it's due to this issue: https://github.com/googleapis/google-cloud-python/pull/9826. After upgrading my six package it's all good now, but maybe it's worth rewording the importerror?
âž• 1
z

Zanie

01/14/2021, 4:10 PM
Hi @Charles Leung, I’m not quite sure where you were getting an error/warning. On a fresh install, I can import the gcp tasks without any messages.
c

Charles Leung

01/14/2021, 4:23 PM
I'm using a jupyter docker image pinned to a specific version from jupyterhub - maybe gcp's dependencies aren't pinned properly then. Just saying this is an issue i came across 🙂
z

Zanie

01/14/2021, 4:30 PM
Ah okay so this import error was being thrown
Copy code
from google.cloud import bigquery
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/__init__.py", line 35, in <module>
    from google.cloud.bigquery.client import Client
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/client.py", line 58, in <module>
    from google.cloud.bigquery import _pandas_helpers
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/_pandas_helpers.py", line 40, in <module>
    from google.cloud.bigquery import schema
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/schema.py", line 17, in <module>
    from six.moves import collections_abc
ImportError: cannot import name collections_abc
And we were hiding it behind a “install gcp silly” message that didn’t apply
c

Charles Leung

01/14/2021, 8:08 PM
yes thats right 🙂 !
z

Zanie

01/14/2021, 10:10 PM
So when we throw this exception we include a 
from
which means it should expose the original import error as well
Copy code
except ImportError as err:
    raise ImportError(
        'Using `prefect.tasks.gcp` requires Prefect to be installed with the "gcp" extra.'
    ) from err
We could add a line before saying “Failed to import the gcp module” as well but scrolling up in the traceback would have exposed the original error and for an edge case like this I think that makes sense. I’m not sure inspecting the original import error would be easy to have work consistently and chaining it with 
from
is the typical pattern here.
3 Views
Powered by