itay livni
05/05/2020, 10:06 PMNoCredentialsError('Unable to locate credentials')
error using Docker Storage (local image), prefect cloud and the s3ResultHandler
When the flow is run locally. There are no credential issues initializing the handler s3_handler = S3ResultHandler(bucket='some-bucket')
To resolve the issue I created a Prefect Secret called "AWS_CREDENTIALS"
and tried
s = Secret("AWS_CREDENTIALS")
aws_scrts = s.get()
s3_handler = S3ResultHandler(bucket='some-bucket', aws_credentials_secret=aws_scrts)
What is the best way to resolve aws credential error? ThanksZachary Hughes
05/05/2020, 10:15 PMS3ResultHandler
your secret values, rather than the name of your Prefect secret. Do you still get the NoCredentialsError
when you provide aws_credentials_secret="NAME_OF_YOUR_SECRET_HERE"
?
Here are a couple of links below for reference, if that's at all helpful.
https://docs.prefect.io/api/latest/engine/result_handlers.html#s3resulthandler
https://github.com/PrefectHQ/prefect/blob/master/src/prefect/engine/result_handlers/s3_result_handler.py#L68itay livni
05/05/2020, 10:45 PM"AWS_CREDENTIALS"
Zachary Hughes
05/05/2020, 10:46 PMitay livni
05/05/2020, 10:47 PMZachary Hughes
05/05/2020, 10:48 PMitay livni
05/05/2020, 10:49 PM'{"ACCESS_KEY": "something", "SECRET_ACCESS_KEY": "something"}'
Zachary Hughes
05/05/2020, 10:54 PMitay livni
05/05/2020, 10:55 PMZachary Hughes
05/05/2020, 11:09 PMitay livni
05/05/2020, 11:17 PMboto3
will fall back on standard AWS rules for authentication."s = Secret("aws_credentials")
aws_scrts = s.get()
s3_handler = S3ResultHandler(bucket='some-bucket', aws_credentials_secret=aws_scrts)
And when I build an image, using flow.register()
and the code above I get this error.
Beginning health checks...
System Version check: OK
Traceback (most recent call last):
File "/root/.prefect/healthcheck.py", line 123, in <module>
flows = cloudpickle_deserialization_check(flow_file_path)
File "/root/.prefect/healthcheck.py", line 39, in cloudpickle_deserialization_check
flows.append(cloudpickle.load(f))
File "/MOC/etl_moc.py", line 48, in <module>
aws_scrts = s.get()
File "/usr/local/lib/python3.7/site-packages/prefect/client/secrets.py", line 139, in get
) from None
ValueError: Local Secret "aws_credentials" was not found.
Zachary Hughes
05/05/2020, 11:29 PMs.get()
, rather than the name of your secret. Can you try passing in the secret name "aws_credentials" like this?
s3_handler = S3ResultHandler(bucket='some-bucket', aws_credentials_secret="aws_credentials")
itay livni
05/06/2020, 12:12 AMZachary Hughes
05/06/2020, 12:18 AM