Laura Vaida
07/01/2021, 8:39 AM/opt/prefect/healthcheck.py:151: UserWarning: Flow uses module which is not importable. Refer to documentation on how to import custom modules <https://docs.prefect.io/api/latest/storage.html#docker>
flows = cloudpickle_deserialization_check(flow_file_paths)
Traceback (most recent call last):
File "/opt/prefect/healthcheck.py", line 151, in <module>
flows = cloudpickle_deserialization_check(flow_file_paths)
File "/opt/prefect/healthcheck.py", line 44, in cloudpickle_deserialization_check
flows.append(cloudpickle.loads(flow_bytes))
ModuleNotFoundError: No module named 'snowflake.sqlalchemy'
Does anybody have experience with that? I tried the following:
flow.storage = Docker(registry_url="xxx", image_name="xxx",
python_dependencies=["pandas", "oauthlib ", "requests", "requests_oauthlib", "oauth2client",
"snowflake-connector-python", "pyarrow", "fastparquet"],
secrets=["GCP_CREDENTIALS"], files={"C:/Users/laura.vaida.000/anaconda3/envs/prefect/Lib/site-packages/sqlalchemy": "/modules/sqlachemy.py"})
Marko Herkaliuk
07/01/2021, 8:53 AM# Generated on: Python 3.6.10
# With snowflake-connector-python version: 2.3.4
asn1crypto==1.4.0
azure-common==1.1.25
azure-core==1.8.2
azure-storage-blob==12.5.0
boto3==1.15.18
botocore==1.18.18
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
cryptography==2.9.2
idna==2.10
isodate==0.6.0
jmespath==0.10.0
msrest==0.6.19
oauthlib==3.1.0
oscrypto==1.2.1
pycparser==2.20
pycryptodomex==3.9.8
PyJWT==1.7.1
pyOpenSSL==19.1.0
python-dateutil==2.8.1
pytz==2020.1
requests==2.23.0
requests-oauthlib==1.3.0
s3transfer==0.3.3
six==1.15.0
urllib3==1.25.11
and i believe you need install sqlalchemy tooLaura Vaida
07/01/2021, 8:54 AMMarko Herkaliuk
07/01/2021, 8:59 AMLaura Vaida
07/01/2021, 9:16 AMflow.storage = Docker(registry_url="xxx", image_name="xxx",
python_dependencies=["pandas", "oauthlib ", "requests", "requests_oauthlib", "oauth2client",
"snowflake-connector-python", "pyarrow", "fastparquet", "asn1crypto==1.4.0",
"azure-common==1.1.25", "azure-core==1.8.2","azure-storage-blob==12.5.0",
"boto3","botocore","certifi==2020.6.20","cffi==1.14.3",
"chardet==3.0.4","cryptography==2.9.2","idna==2.10","isodate==0.6.0",
"jmespath==0.10.0","msrest==0.6.19","oauthlib==3.1.0","oscrypto==1.2.1",
"pycparser==2.20","pycryptodomex==3.9.8","PyJWT==1.7.1","pyOpenSSL==19.1.0",
"python-dateutil==2.8.1","pytz==2020.1","requests==2.23.0","requests-oauthlib==1.3.0",
"s3transfer==0.3.3","six==1.15.0","urllib3==1.25.11"],secrets=["GCP_CREDENTIALS"],
files={"C:/Users/laura.vaida.000/anaconda3/envs/prefect/Lib/site-packages/sqlalchemy": "/modules/sqlachemy.py"})
Marko Herkaliuk
07/01/2021, 9:24 AMMarko Herkaliuk
07/01/2021, 9:25 AMsnowflake-sqlalchemy
Laura Vaida
07/01/2021, 1:04 PMMarko Herkaliuk
07/01/2021, 2:46 PM