https://prefect.io logo
Join Slack
Powered by
Hi - I am encountering a botocore `NoCredentialsEr...
# prefect-community
i
Hi - I am encountering a botocore 
NoCredentialsError('Unable to locate credentials')
error using Docker Storage (local image), prefect cloud and the 
s3ResultHandler
When the flow is run locally. There are no credential issues initializing the handler 
s3_handler = S3ResultHandler(bucket='some-bucket')
To resolve the issue I created a Prefect Secret called 
"AWS_CREDENTIALS"
and tried
Copy code
s = Secret("AWS_CREDENTIALS")
aws_scrts = s.get()
s3_handler = S3ResultHandler(bucket='some-bucket', aws_credentials_secret=aws_scrts)
What is the best way to resolve aws credential error? Thanks
z
Hi @itay livni, taking a look now! Will get back to you with an answer ASAP.
It looks like you might be giving the 
S3ResultHandler
your secret values, rather than the name of your Prefect secret. Do you still get the 
NoCredentialsError
when you provide 
aws_credentials_secret="NAME_OF_YOUR_SECRET_HERE"
? Here are a couple of links below for reference, if that's at all helpful. https://docs.prefect.io/api/latest/engine/result_handlers.html#s3resulthandler https://github.com/PrefectHQ/prefect/blob/master/src/prefect/engine/result_handlers/s3_result_handler.py#L68
i
@Zachary Hughes Just to be clear the AWS Prefect Secret name has to be all caps? 
"AWS_CREDENTIALS"
z
Not necessarily-- it just needs to be the name of your secret. I just used all-caps to denote that this was a portion of code that needed to be replaced. 🙂
i
So something else is going on. Because this flow was running previously in the same enviroment.. using different version of prefect
z
Hmm... do you recall what version of Prefect you were on, and what version you're on now?
i
well I am on 0.10.6
and the secret should look like this?
Copy code
'{"ACCESS_KEY": "something", "SECRET_ACCESS_KEY": "something"}'
z
I believe so!
i
Strange I did not need to muck around with this previously
z
I can ask around to see if folks know of any changes that could have impacted this! It depends on your setup, but I'm also wondering if boto's picking up changes in your environment/config, as the S3ResultHandler defaults to standard boto3 auth if you don't provide a secret.
Out of curiosity, how were you providing AWS creds previously?
i
I was not. It was magically happening behind the scene "If not provided, 
boto3
 will fall back on standard AWS rules for authentication."
Now - That stopped working
Copy code
s = Secret("aws_credentials")
aws_scrts = s.get()

s3_handler = S3ResultHandler(bucket='some-bucket', aws_credentials_secret=aws_scrts)
And when I build an image, using 
flow.register()
and the code above I get this error.
Copy code
Beginning health checks...
System Version check: OK
Traceback (most recent call last):
  File "/root/.prefect/healthcheck.py", line 123, in <module>
    flows = cloudpickle_deserialization_check(flow_file_path)
  File "/root/.prefect/healthcheck.py", line 39, in cloudpickle_deserialization_check
    flows.append(cloudpickle.load(f))
  File "/MOC/etl_moc.py", line 48, in <module>
    aws_scrts = s.get()
  File "/usr/local/lib/python3.7/site-packages/prefect/client/secrets.py", line 139, in get
    ) from None
ValueError: Local Secret "aws_credentials" was not found.
and the secret does exist:
z
It looks like you're passing in the value of 
s.get()
, rather than the name of your secret. Can you try passing in the secret name "aws_credentials" like this? 
s3_handler = S3ResultHandler(bucket='some-bucket', aws_credentials_secret="aws_credentials")
😁 1
i
@Zachary Hughes - For closure. (1) Why I "suddenly" need to use a prefect aws configuration is a mystery. (2) Using prefect secret works.
Thank you for your patience!
z
No patience required-- just glad it's working for you! 🙂 Haven't been able to isolate any core changes that might have prompted this shift, but I'll be sure to keep an eye out.
Open in Slack
PreviousNext
Powered by