c

    Charles Leung

    1 year ago
    Hey prefect team, just wanted to let you know a heads up on installing GCP extras with pip - 🙂 i ran into this issue where it kept saying it needed to be installed as a prefect extra, but realized after i removed the try/except in the _init that_ it's due to this issue: https://github.com/googleapis/google-cloud-python/pull/9826. After upgrading my six package it's all good now, but maybe it's worth rewording the importerror?
    Michael Adkins

    Michael Adkins

    1 year ago
    Hi @Charles Leung, I’m not quite sure where you were getting an error/warning. On a fresh install, I can import the gcp tasks without any messages.
    c

    Charles Leung

    1 year ago
    I'm using a jupyter docker image pinned to a specific version from jupyterhub - maybe gcp's dependencies aren't pinned properly then. Just saying this is an issue i came across 🙂
    Michael Adkins

    Michael Adkins

    1 year ago
    Ah okay so this import error was being thrown
    from google.cloud import bigquery
      File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/__init__.py", line 35, in <module>
        from google.cloud.bigquery.client import Client
      File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/client.py", line 58, in <module>
        from google.cloud.bigquery import _pandas_helpers
      File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/_pandas_helpers.py", line 40, in <module>
        from google.cloud.bigquery import schema
      File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/schema.py", line 17, in <module>
        from six.moves import collections_abc
    ImportError: cannot import name collections_abc
    And we were hiding it behind a “install gcp silly” message that didn’t apply
    c

    Charles Leung

    1 year ago
    yes thats right 🙂 !
    Michael Adkins

    Michael Adkins

    1 year ago
    So when we throw this exception we include a
    from
    which means it should expose the original import error as well
    except ImportError as err:
        raise ImportError(
            'Using `prefect.tasks.gcp` requires Prefect to be installed with the "gcp" extra.'
        ) from err
    We could add a line before saying “Failed to import the gcp module” as well but scrolling up in the traceback would have exposed the original error and for an edge case like this I think that makes sense. I’m not sure inspecting the original import error would be easy to have work consistently and chaining it with
    from
    is the typical pattern here.