Lance Cuthbert
09/28/2022, 6:09 PMMason Menges
09/28/2022, 6:41 PMLance Cuthbert
09/28/2022, 6:43 PM[2022-09-28 18:02:59+0000] ERROR - prefect.S3 | Error downloading Flow from S3: deprecated() got an unexpected keyword argument 'name'
deprecated() got an unexpected keyword argument 'name'
/usr/local/lib/python3.7/site-packages/prefect/client/client.py:177: UserWarning: Client was created with an API token configured for authentication. API tokens are deprecated, please use API keys instead.
"Client was created with an API token configured for authentication. "
Traceback (most recent call last):
File "/usr/local/bin/prefect", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/cli/execute.py", line 96, in flow_run
raise exc
File "/usr/local/lib/python3.7/site-packages/prefect/cli/execute.py", line 73, in flow_run
flow = storage.get_flow(flow_data.name)
File "/usr/local/lib/python3.7/site-packages/prefect/storage/s3.py", line 89, in get_flow
obj = self._boto3_client.get_object(Bucket=self.bucket, Key=key)
File "/usr/local/lib/python3.7/site-packages/prefect/storage/s3.py", line 219, in _boto3_client
from prefect.utilities.aws import get_boto_client
File "/usr/local/lib/python3.7/site-packages/prefect/utilities/aws.py", line 10, in <module>
import boto3
File "/usr/local/lib/python3.7/site-packages/boto3/__init__.py", line 16, in <module>
from boto3.session import Session
File "/usr/local/lib/python3.7/site-packages/boto3/session.py", line 17, in <module>
import botocore.session
File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 29, in <module>
import botocore.credentials
File "/usr/local/lib/python3.7/site-packages/botocore/credentials.py", line 35, in <module>
from botocore.config import Config
File "/usr/local/lib/python3.7/site-packages/botocore/config.py", line 16, in <module>
from botocore.endpoint import DEFAULT_TIMEOUT, MAX_POOL_CONNECTIONS
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 22, in <module>
from botocore.awsrequest import create_request_object
File "/usr/local/lib/python3.7/site-packages/botocore/awsrequest.py", line 24, in <module>
import botocore.utils
File "/usr/local/lib/python3.7/site-packages/botocore/utils.py", line 32, in <module>
import botocore.httpsession
File "/usr/local/lib/python3.7/site-packages/botocore/httpsession.py", line 28, in <module>
from urllib3.contrib.pyopenssl import orig_util_SSLContext as SSLContext
File "/usr/local/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 50, in <module>
import OpenSSL.SSL
File "/usr/local/lib/python3.7/site-packages/OpenSSL/__init__.py", line 8, in <module>
from OpenSSL import SSL, crypto
File "/usr/local/lib/python3.7/site-packages/OpenSSL/SSL.py", line 19, in <module>
from OpenSSL.crypto import (
File "/usr/local/lib/python3.7/site-packages/OpenSSL/crypto.py", line 3232, in <module>
name="load_pkcs7_data",
TypeError: deprecated() got an unexpected keyword argument 'name'
`Lance Cuthbert
09/28/2022, 6:50 PMMason Menges
09/28/2022, 6:58 PMLance Cuthbert
09/28/2022, 7:02 PMLance Cuthbert
09/28/2022, 7:06 PMUserWarning: Client was created with an API token configured for authentication. API tokens are deprecated, please use API keys instead.
"Client was created with an API token configured for authentication. "
Mason Menges
09/28/2022, 8:17 PMstorage = S3(bucket="********bucket", key="test_flow")
@task
def some_task():
print("test")
with Flow(name="test_flow",
storage=storage,
run_config=UniversalRun(env=dict(AWS_ACCESS_KEY_ID=Secret("aws_personal_key_id").get(),
AWS_SECRET_ACCESS_KEY=Secret("aws_personal_api_key").get()))) as flow:
some_task()
flow.register(project_name="Local-test")
with prefect version 1.3.0Lance Cuthbert
09/28/2022, 8:27 PMLance Cuthbert
09/28/2022, 8:37 PMprefect agent docker start -t EdYAD_SW9wvRsPfsf42yQQ -n DockerAgent -l <http://prefect.segmint.com|prefect.segmint.com> --volume /opt/segmint:/opt/segmint --show-flow-logs
____ __ _ _ _
| _ \ _ __ ___ / _| ___ ___| |_ / \ __ _ ___ _ __ | |_
| |_) | '__/ _ \ |_ / _ \/ __| __| / _ \ / _` |/ _ \ '_ \| __|
| __/| | | __/ _| __/ (__| |_ / ___ \ (_| | __/ | | | |_
|_| |_| \___|_| \___|\___|\__| /_/ \_\__, |\___|_| |_|\__|
|___/
[2022-09-28 20:34:21,989] INFO - DockerAgent | Starting DockerAgent with labels ['<http://prefect.segmint.com|prefect.segmint.com>']
[2022-09-28 20:34:21,989] INFO - DockerAgent | Agent documentation can be found at <https://docs.prefect.io/orchestration/>
[2022-09-28 20:34:21,989] INFO - DockerAgent | Agent connecting to the Prefect API at <https://api.prefect.io>
[2022-09-28 20:34:22,076] INFO - DockerAgent | Waiting for flow runs...
[2022-09-28 20:34:22,300] INFO - DockerAgent | Found 1 flow run(s) to submit for execution.
[2022-09-28 20:34:22,437] INFO - DockerAgent | Deploying flow run 437b9710-93ce-49ca-a014-16495d381681
[2022-09-28 20:34:22,439] INFO - DockerAgent | Pulling image prefecthq/prefect:1.3.0-python3.7...
[2022-09-28 20:34:37,925] INFO - DockerAgent | Successfully pulled image prefecthq/prefect:1.3.0-python3.7...
+pip install snowflake-connector-python
Collecting snowflake-connector-python
Downloading snowflake_connector_python-2.8.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (21.3 MB)
Collecting oscrypto<2.0.0
Downloading oscrypto-1.3.0-py2.py3-none-any.whl (194 kB)
Requirement already satisfied: cffi<2.0.0,>=1.9 in /usr/local/lib/python3.7/site-packages (from snowflake-connector-python) (1.15.1)
Requirement already satisfied: charset-normalizer<3,>=2 in /usr/local/lib/python3.7/site-packages (from snowflake-connector-python) (2.1.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/site-packages (from snowflake-connector-python) (3.3)
Requirement already satisfied: typing-extensions<5,>=4.3 in /usr/local/lib/python3.7/site-packages (from snowflake-connector-python) (4.3.0)
Requirement already satisfied: requests<3.0.0 in /usr/local/lib/python3.7/site-packages (from snowflake-connector-python) (2.28.1)
Collecting filelock<4,>=3.5
Downloading filelock-3.8.0-py3-none-any.whl (10 kB)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/site-packages (from snowflake-connector-python) (2022.6.15)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/site-packages (from snowflake-connector-python) (1.26.11)
Collecting pycryptodomex!=3.5.0,<4.0.0,>=3.2
Downloading pycryptodomex-3.15.0-cp35-abi3-manylinux2010_x86_64.whl (2.3 MB)
Requirement already satisfied: setuptools>34.0.0 in /usr/local/lib/python3.7/site-packages (from snowflake-connector-python) (57.5.0)
Requirement already satisfied: pyjwt<3.0.0 in /usr/local/lib/python3.7/site-packages (from snowflake-connector-python) (2.4.0)
Collecting cryptography<37.0.0,>=3.1.0
Downloading cryptography-36.0.2-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.7 MB)
Collecting asn1crypto<2.0.0,>0.24.0
Downloading asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)
Requirement already satisfied: pytz in /usr/local/lib/python3.7/site-packages (from snowflake-connector-python) (2022.1)
Collecting pyOpenSSL<23.0.0,>=16.2.0
Downloading pyOpenSSL-22.1.0-py3-none-any.whl (57 kB)
Requirement already satisfied: pycparser in /usr/local/lib/python3.7/site-packages (from cffi<2.0.0,>=1.9->snowflake-connector-python) (2.21)
Installing collected packages: asn1crypto, oscrypto, filelock, pycryptodomex, cryptography, pyOpenSSL, snowflake-connector-python
Attempting uninstall: cryptography
Found existing installation: cryptography 37.0.4
Uninstalling cryptography-37.0.4:
Successfully uninstalled cryptography-37.0.4
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.
We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
pyopenssl 22.1.0 requires cryptography<39,>=38.0.0, but you'll have cryptography 36.0.2 which is incompatible.
Successfully installed asn1crypto-1.5.1 cryptography-36.0.2 filelock-3.8.0 oscrypto-1.3.0 pyOpenSSL-22.1.0 pycryptodomex-3.15.0 snowflake-connector-python-2.8.0
WARNING: You are using pip version 20.2.4; however, version 22.2.2 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
Traceback (most recent call last):
File "/usr/local/bin/prefect", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/cli/execute.py", line 53, in flow_run
result = client.graphql(query)
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 464, in graphql
raise ClientError(result["errors"])
prefect.exceptions.ClientError: [{'path': ['flow_run'], 'message': 'Missing Authorization header in JWT authentication mode', 'extensions': {'path': '$', 'code': 'invalid-headers', 'exception': {'message': 'Missing Authorization header in JWT authentication mode'}}}]
Mason Menges
09/28/2022, 8:38 PMLance Cuthbert
09/29/2022, 4:13 PMMason Menges
09/29/2022, 4:16 PMLance Cuthbert
09/29/2022, 4:18 PMMason Menges
09/29/2022, 4:19 PMBianca Hoch
09/29/2022, 6:02 PMLance Cuthbert
09/29/2022, 6:25 PMenv={"EXTRA_PIP_PACKAGES": "snowflake-connector-python==2.4.3"}
Bianca Hoch
09/29/2022, 6:27 PM