Hi All I m running into some issues with my CI build running Prefect Community #ask-community

Hi All I'm running into some issues with my CI bui...

Belal Aboabdo

11/08/2021, 9:32 PM

Hi All I'm running into some issues with my CI build running on ECS. Here's the traceback. I'm seeing an error related to not installing the prefect "aws" extra but it is listed in my flows requirements.

Kevin Kho

11/08/2021, 9:35 PM

Hey @Belal Aboabdo, could you move the traceback to the thread? to not crowd the main channel? You could install boto3 separately. Are you using pip here?

Belal Aboabdo

11/08/2021, 9:36 PM

Copy code

Installing: awswrangler==2.11, pandas, prefect[ge], prefect[aws], great_expectations==0.12, requests, s3fs==0.4, smartystreets_python_sdk==4.7.2
+ python -m prefect register -p data_flows/flows/flow_name.py --label dev --project dev
Collecting flows...
Error loading 'data_flows/flows/flow_name.py':
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/site-packages/prefect/tasks/aws/__init__.py", line 7, in <module>
    from prefect.tasks.aws.s3 import S3Download, S3Upload, S3List
    File "/usr/local/lib/python3.9/site-packages/prefect/tasks/aws/s3.py", line 7, in <module>
    from prefect.utilities.aws import get_boto_client
    File "/usr/local/lib/python3.9/site-packages/prefect/utilities/aws.py", line 10, in <module>
    import boto3
    File "/usr/local/lib/python3.9/site-packages/boto3/__init__.py", line 16, in <module>
    from boto3.session import Session
    File "/usr/local/lib/python3.9/site-packages/boto3/session.py", line 17, in <module>
    import botocore.session
    File "/usr/local/lib/python3.9/site-packages/botocore/session.py", line 29, in <module>
    import botocore.credentials
    File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 34, in <module>
    from botocore.config import Config
    File "/usr/local/lib/python3.9/site-packages/botocore/config.py", line 16, in <module>
    from botocore.endpoint import DEFAULT_TIMEOUT, MAX_POOL_CONNECTIONS
    File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 22, in <module>
    from botocore.awsrequest import create_request_object
    File "/usr/local/lib/python3.9/site-packages/botocore/awsrequest.py", line 24, in <module>
    import botocore.utils
    File "/usr/local/lib/python3.9/site-packages/botocore/utils.py", line 32, in <module>
    import botocore.httpsession
    File "/usr/local/lib/python3.9/site-packages/botocore/httpsession.py", line 10, in <module>
    from urllib3.util.ssl_ import (
  ImportError: cannot import name 'OP_NO_TICKET' from 'urllib3.util.ssl_' (/root/.local/lib/python3.9/site-packages/urllib3/util/ssl_.py)
 
The above exception was the direct cause of the following exception:
 
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/site-packages/prefect/cli/build_register.py", line 134, in load_flows_from_script
    namespace = runpy.run_path(abs_path, run_name="<flow>")
    File "/usr/local/lib/python3.9/runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
    File "/usr/local/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
    File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
    File "/src/data_flows/flows/flow_name.py", line 23, in <module>
    from data_flows.utils.aws_tasks import AWSSecretFetcher, S3MostRecentUpdated
    File "/src/data_flows/utils/aws_tasks.py", line 5, in <module>
    from prefect.tasks.aws.s3 import S3List, S3Upload
    File "/usr/local/lib/python3.9/site-packages/prefect/tasks/aws/__init__.py", line 19, in <module>
    raise ImportError(
  ImportError: Using `prefect.tasks.aws` requires Prefect to be installed with the "aws" extra.

Kevin Kho

11/08/2021, 9:36 PM

Thank you!

Belal Aboabdo

11/08/2021, 9:37 PM

The following are installed separately using pip

Copy code

datadog>=0.39.0
boto3>=1.16.8
pytest>=6.2
PyYAML>=5.4
prefect==0.15.6
pip-tools

Kevin Kho

11/08/2021, 9:40 PM

Just clarifying, is this installation traceback on the CI/CD environment and this is during registration right?

Belal Aboabdo

11/08/2021, 9:41 PM

That's correct

Kevin Kho

11/08/2021, 9:44 PM

I think that error is just the result of a try-except. Prefect requirements says

"boto3 >= 1.9, < 2.0"

so this might work if you use a later version? Hard to tell though why the pip is not installing. What is the OS of your CI/CD environment?

Belal Aboabdo

11/08/2021, 9:53 PM

I can try that out. We're using the python 3.9 image from here

Copy code

FROM python:3.9

https://hub.docker.com/_/python

Kevin Kho

11/08/2021, 10:00 PM

Can you try

prefect[ge, aws]

Belal Aboabdo

11/08/2021, 10:39 PM

I'm getting the same error as above here's the full updated pip requirements

Copy code

Successfully installed Send2Trash-1.8.0 altair-4.1.0 appdirs-1.4.4 argon2-cffi-21.1.0 asn1crypto-1.4.0 awswrangler-2.11.0 backcall-0.2.0 beautifulsoup4-4.10.0 black-19.10b0 bleach-4.1.0 cffi-1.15.0 chardet-3.0.4 debugpy-1.5.1 decorator-5.1.0 defusedxml-0.7.1 entrypoints-0.3 et-xmlfile-1.1.0 great-expectations-0.12.0 importlib-metadata-4.8.1 ipykernel-6.5.0 ipython-7.29.0 ipython-genutils-0.2.0 ipywidgets-7.6.5 jedi-0.18.0 jsonpatch-1.32 jsonpointer-2.2 jsonschema-4.2.1 jupyter-client-7.0.6 jupyter-core-4.9.1 jupyterlab-pygments-0.1.2 jupyterlab-widgets-1.0.2 lxml-4.6.4 matplotlib-inline-0.1.3 mistune-0.8.4 nbclient-0.5.5 nbconvert-6.2.0 nbformat-5.1.3 nest-asyncio-1.5.1 notebook-6.4.5 numpy-1.21.4 openpyxl-3.0.9 pandas-1.3.4 pandocfilters-1.5.0 parso-0.8.2 pathspec-0.9.0 pexpect-4.8.0 pg8000-1.21.3 pickleshare-0.7.5 prometheus-client-0.12.0 prompt-toolkit-3.0.22 ptyprocess-0.7.0 pyarrow-5.0.0 pycparser-2.21 pygments-2.10.0 pymysql-1.0.2 pyrsistent-0.18.0 pytz-deprecation-shim-0.1.0.post0 pyzmq-22.3.0 redshift-connector-2.0.889 regex-2021.11.2 requests-2.23.0 ruamel.yaml-0.17.17 ruamel.yaml.clib-0.2.6 s3fs-0.4.0 scipy-1.7.2 scramp-1.4.1 smartystreets-python-sdk-4.7.2 soupsieve-2.3 termcolor-1.1.0 terminado-0.12.1 testpath-0.5.0 traitlets-5.1.1 typed-ast-1.4.3 tzdata-2021.5 tzlocal-4.1 urllib3-1.25.11 wcwidth-0.2.5 webencodings-0.5.1 widgetsnbextension-3.5.2 zipp-3.6.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: <https://pip.pypa.io/warnings/venv>
Installing: awswrangler==2.11, pandas, prefect[ge, aws], great_expectations==0.12, requests, s3fs==0.4, smartystreets_python_sdk==4.7.2
+ python -m prefect register -p data_flows/flows/pgl_import.py --label dev --project dev

Belal Aboabdo

11/09/2021, 12:27 AM

Just following up on this thread if there's any more info I can provide. This began failing today.

Kevin Kho

11/09/2021, 12:31 AM

Hey sorry I’ve been in a call. Am very confused it’s not appearing in the

successfully installed

portion. Does even installing boto3 fail?

Belal Aboabdo

11/09/2021, 12:40 AM

No worries. I believe it is being installed by awswrangler. This is only occurring with the flows that require the prefect[ge] extra.

Belal Aboabdo

11/09/2021, 12:48 AM

pinning boto3 to 1.20.0 resolved my issue thanks for all the help @Kevin Kho

Kevin Kho

11/09/2021, 12:48 AM

I think this is your issue? Do you upgrade pip?

✅ 1

Belal Aboabdo

11/09/2021, 12:49 AM

I didn't need to upgrade pip just boto3

Kevin Kho

11/09/2021, 12:50 AM

No I know but pip itself struggles if you do

prefect[ge], prefect[aws]

. It doesn’t install the aws one.

Kevin Kho

11/09/2021, 12:50 AM

Which is what is detailed in the issue

Belal Aboabdo

11/09/2021, 12:51 AM

Ah makes sense

Kevin Kho

11/09/2021, 12:56 AM

I dunno if pip fixed it. I went through the issue and it looks like they didnt

Belal Aboabdo

11/10/2021, 9:22 PM

Weird I previously resolved the issue by setting

boto3==1.20.0

but the issue is now reoccurring with the great expectations prefect extra.

Copy code

Successfully installed Send2Trash-1.8.0 altair-4.1.0 appdirs-1.4.4 argon2-cffi-21.1.0 asn1crypto-1.4.0 awswrangler-2.8.0 backcall-0.2.0 beautifulsoup4-4.10.0 black-19.10b0 bleach-4.1.0 cffi-1.15.0 chardet-3.0.4 debugpy-1.5.1 decorator-5.1.0 defusedxml-0.7.1 entrypoints-0.3 et-xmlfile-1.1.0 great-expectations-0.12.0 importlib-metadata-4.8.2 ipykernel-6.5.0 ipython-7.29.0 ipython-genutils-0.2.0 ipywidgets-7.6.5 jedi-0.18.0 jsonpatch-1.32 jsonpointer-2.2 jsonschema-4.2.1 jupyter-client-7.0.6 jupyter-core-4.9.1 jupyterlab-pygments-0.1.2 jupyterlab-widgets-1.0.2 lxml-4.6.4 matplotlib-inline-0.1.3 mistune-0.8.4 nbclient-0.5.5 nbconvert-6.2.0 nbformat-5.1.3 nest-asyncio-1.5.1 notebook-6.4.5 numpy-1.20.3 openpyxl-3.0.9 pandas-1.2.0 pandocfilters-1.5.0 parso-0.8.2 pathspec-0.9.0 pexpect-4.8.0 pg8000-1.19.5 pickleshare-0.7.5 prometheus-client-0.12.0 prompt-toolkit-3.0.22 ptyprocess-0.7.0 pyarrow-4.0.1 pycparser-2.21 pygments-2.10.0 pymysql-1.0.2 pyrsistent-0.18.0 pytz-deprecation-shim-0.1.0.post0 pyzmq-22.3.0 redshift-connector-2.0.889 regex-2021.11.10 requests-2.23.0 ruamel.yaml-0.17.17 ruamel.yaml.clib-0.2.6 scipy-1.7.2 scramp-1.4.0 soupsieve-2.3 termcolor-1.1.0 terminado-0.12.1 testpath-0.5.0 traitlets-5.1.1 typed-ast-1.4.3 tzdata-2021.5 tzlocal-4.1 urllib3-1.25.11 wcwidth-0.2.5 webencodings-0.5.1 widgetsnbextension-3.5.2 zipp-3.6.0
Installing: pandas==1.2, awswrangler==2.8, prefect[aws,ge], great_expectations==0.12, requests
+ python -m prefect register -p data_flows/flows/cms_import.py --label dev --project dev
Collecting flows...
Error loading 'data_flows/flows/cms_import.py':
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/site-packages/prefect/tasks/great_expectations/__init__.py", line 8, in <module>
    from prefect.tasks.great_expectations.checkpoints import (
    File "/usr/local/lib/python3.9/site-packages/prefect/tasks/great_expectations/checkpoints.py", line 20, in <module>
    import great_expectations as ge
    File "/root/.local/lib/python3.9/site-packages/great_expectations/__init__.py", line 7, in <module>
    from great_expectations.data_context import DataContext
    File "/root/.local/lib/python3.9/site-packages/great_expectations/data_context/__init__.py", line 3, in <module>
    from .data_context import BaseDataContext, DataContext, ExplorerDataContext
    File "/root/.local/lib/python3.9/site-packages/great_expectations/data_context/data_context.py", line 29, in <module>
    from great_expectations.core.usage_statistics.usage_statistics import (
    File "/root/.local/lib/python3.9/site-packages/great_expectations/core/usage_statistics/usage_statistics.py", line 17, in <module>
    from great_expectations.core.usage_statistics.anonymizers.anonymizer import Anonymizer
    File "/root/.local/lib/python3.9/site-packages/great_expectations/core/usage_statistics/anonymizers/anonymizer.py", line 4, in <module>
    from great_expectations.util import load_class
    File "/root/.local/lib/python3.9/site-packages/great_expectations/util.py", line 12, in <module>
    import black
    File "/root/.local/lib/python3.9/site-packages/black.py", line 45, in <module>
    from typed_ast import ast3, ast27
    File "/root/.local/lib/python3.9/site-packages/typed_ast/ast3.py", line 40, in <module>
    from typed_ast import _ast3
  ImportError: /root/.local/lib/python3.9/site-packages/typed_ast/_ast3.cpython-39-x86_64-linux-gnu.so: undefined symbol: _PyUnicode_DecodeUnicodeEscape
 
The above exception was the direct cause of the following exception:
 
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/site-packages/prefect/cli/build_register.py", line 134, in load_flows_from_script
    namespace = runpy.run_path(abs_path, run_name="<flow>")
    File "/usr/local/lib/python3.9/runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
    File "/usr/local/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
    File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
    File "/src/data_flows/flows/cms_import.py", line 14, in <module>
    from prefect.tasks.great_expectations.checkpoints import RunGreatExpectationsValidation
    File "/usr/local/lib/python3.9/site-packages/prefect/tasks/great_expectations/__init__.py", line 12, in <module>
    raise ImportError(
  ImportError: Using `prefect.tasks.great_expectations` requires Prefect to be installed with the "ge" extra.

Kevin Kho

11/10/2021, 9:30 PM

Seems like your installation is successful. There is just an error

ImportError: /root/.local/lib/python3.9/site-packages/typed_ast/_ast3.cpython-39-x86_64-linux-gnu.so: undefined symbol: _PyUnicode_DecodeUnicodeEscape

Belal Aboabdo

11/10/2021, 9:33 PM

What would that refer to? Is it not related to this import error?

Copy code

ImportError: Using `prefect.tasks.great_expectations` requires Prefect to be installed with the "ge" extra.

Kevin Kho

11/10/2021, 9:35 PM

That is just the result of a try catch here . So it will print for all errors that happen from the import and it looks like there is another import error really happening here

Belal Aboabdo

11/10/2021, 10:07 PM

Do you know which version of

great_expectations

is supported by the prefect[ge] extra?

Kevin Kho

11/10/2021, 10:09 PM

I think you have the right one . I don’t know if this one is compatible with Python 3.9 though

Belal Aboabdo

11/10/2021, 10:21 PM

Do you know what a possible work around for this issue?

Kevin Kho

11/10/2021, 10:23 PM

I don’t and didn’t see a lot when I searched but I imagine you can try to recreate the environment on local and see if that import succeeds. I suspect Python 3.8 would be more stable

👍 1

12 Views

Open in Slack

Previous Next