https://prefect.io logo
Join Slack
Powered by
Hi all, I'm trying to deploy a flow that uses `pre...
# ask-community
d
Hi all, I'm trying to deploy a flow that uses 
prefect-snowflake
, however I received this error message indicating one of the subpackages is missing something:
Copy code
AttributeError: cffi library '_openssl' has no function, constant or global variable named 'Cryptography_HAS_OPENSSL_CLEANUP'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/deploy.py", line 80, in <module>
    flow = load_flow_from_script(path=f"{working_dir}/{config['flow_path']}", flow_name=config['flow_name'].replace('_', '-'))
  File "/usr/local/lib/python3.8/site-packages/prefect/flows.py", line 794, in load_flow_from_script
    load_flows_from_script(path),
  File "/usr/local/lib/python3.8/site-packages/prefect/flows.py", line 770, in load_flows_from_script
    return registry_from_script(path).get_instances(Flow)
  File "/usr/local/lib/python3.8/site-packages/prefect/context.py", line 449, in registry_from_script
    load_script_as_module(path)
  File "/usr/local/lib/python3.8/site-packages/prefect/utilities/importtools.py", line 164, in load_script_as_module
    raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at '/github/workspace/etl_trigger/trigger_candidate.py' encountered an exception: AttributeError("cffi library '_openssl' has no function, constant or global variable named 'Cryptography_HAS_OPENSSL_CLEANUP'")
Is anyone else seeing this issue?
The full stack trace:
Copy code
Extracting flow from etl_trigger/trigger_candidate.py
Traceback (most recent call last):
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/github/workspace/etl_trigger/trigger_candidate.py", line 8, in <module>
    from prefect_snowflake import SnowflakeConnector, SnowflakeCredentials  # type: ignore
  File "/usr/local/lib/python3.8/site-packages/prefect_snowflake/__init__.py", line 2, in <module>
    from prefect_snowflake.credentials import SnowflakeCredentials  # noqa
  File "/usr/local/lib/python3.8/site-packages/prefect_snowflake/credentials.py", line 21, in <module>
    import snowflake.connector
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/__init__.py", line 17, in <module>
    from .connection import SnowflakeConnection
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/connection.py", line 27, in <module>
    from . import errors, proxy
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/errors.py", line 18, in <module>
    from .telemetry_oob import TelemetryService
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/telemetry_oob.py", line 21, in <module>
    from .vendored import requests
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/vendored/requests/__init__.py", line 119, in <module>
    from ..urllib3.contrib import pyopenssl
  File "/usr/local/lib/python3.8/site-packages/snowflake/connector/vendored/urllib3/contrib/pyopenssl.py", line 50, in <module>
    import OpenSSL.crypto
  File "/usr/local/lib/python3.8/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import SSL, crypto
  File "/usr/local/lib/python3.8/site-packages/OpenSSL/SSL.py", line 9, in <module>
    from OpenSSL._util import (
  File "/usr/local/lib/python3.8/site-packages/OpenSSL/_util.py", line 6, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 179, in <module>
    Binding.init_static_locks()
  File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 146, in init_static_locks
    cls._ensure_ffi_initialized()
  File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 117, in _ensure_ffi_initialized
    cls.lib = build_conditional_library(
  File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 59, in build_conditional_library
    if not getattr(lib, condition):
AttributeError: cffi library '_openssl' has no function, constant or global variable named 'Cryptography_HAS_OPENSSL_CLEANUP'
n
Hi @David Beck - what is your os? this has worked for me in some situations in the past when seeing stuff like this
Copy code
pip uninstall cryptography pyopenssl
pip install cryptography pyopenssl
d
It's a linux docker container. Did you uninstall/reinstall before or after installing the prefect-snowflake package?
n
hmm perhaps its a matter of base image choice then, what have you started from?
or could you share your whole Dockerfile?
d
Here's our docker file:
Copy code
FROM python:3.8-buster

COPY . .

# configure apt-get to be able to find msodbcsql17
RUN echo msodbcsql17 msodbcsql/ACCEPT_EULA boolean true | debconf-set-selections
RUN curl <https://packages.microsoft.com/keys/microsoft.asc> | apt-key add -
RUN curl <https://packages.microsoft.com/config/debian/10/prod.list> > /etc/apt/sources.list.d/mssql-release.list

# manually install unixodbc and msodbc to allow python to connect to SQL Server
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
    && apt-get -y install --no-install-recommends unixodbc-dev msodbcsql17

RUN apt update
RUN apt install apt-transport-https ca-certificates curl gnupg2 software-properties-common -y
RUN curl -fsSL <https://download.docker.com/linux/debian/gpg> | apt-key add -
RUN add-apt-repository "deb [arch=amd64] <https://download.docker.com/linux/debian> $(lsb_release -cs) stable"
RUN apt update
RUN apt-cache policy docker-ce
RUN apt install docker-ce -y

# install python dependencies for the job
RUN pip3 --disable-pip-version-check --no-cache-dir install -r ./requirements.txt
RUN chmod +x ./bootstrap.sh
ENTRYPOINT ["/bootstrap.sh"]
our deploy.py is run off the bootstrap.sh. We install all pip dependencies just before deployment. In most cases, this works fine aside from this issue
n
hmm - yeah I've seen plenty of weirdness related to python's cryptography / pyopenssl package on linux before what version of prefect are you installing from your 
requirements.txt
?
perhaps you could try having this in your 
requirements.txt
?
Copy code
cryptography >= 36.0.1
d
Copy code
Successfully installed asn1crypto-1.5.1 cryptography-40.0.2 filelock-3.12.0 oscrypto-1.3.0 prefect-snowflake-0.26.1 pycryptodomex-3.18.0 snowflake-connector-python-3.0.4 sortedcontainers-2.4.0
runner shows it is installing 40.0.2
the prefect version that is on our runner is 2.10.6 I believe
n
hmm could I see the deploy file? depending on your infra, your agent/worker will use different deps than your runtime (unless you're using Process infra)
d
I PMed you our deploy script
10 Views
Open in Slack
PreviousNext
Powered by