Hey guys ran into an issue with prefect 0.12.4 It seems the context secret manager for AWS is no longer automatically loading local

AWS_CREDENTIALS

to dict as is did in

0.12.3

. eg.

cat <<EOF > ~/.prefect/config.toml
[context.secrets]
AWS_CREDENTIALS='{"ACCESS_KEY": "${AWS_ACCESS_KEY_ID}", "SECRET_ACCESS_KEY": "${AWS_SECRET_ACCESS_KEY}"}'
DASK_GATEWAY_URL="${DASK_GATEWAY_URL}"
SLACK_WEBHOOK_URL="${SLACK_WEBHOOK_URL}"

[flows]
CHECKPOINTING=true
EOF

...

>> flow.storage = S3(<>)
>> flow.register('Report')

/usr/local/lib/python3.7/site-packages/prefect/utilities/aws.py in get_boto_client(resource, credentials, use_session, **kwargs)
     36     else:
     37         ctx_credentials = prefect.context.get("secrets", {}).get("AWS_CREDENTIALS", {})
---> 38         aws_access_key = ctx_credentials.get("ACCESS_KEY")
     39         aws_secret_access_key = ctx_credentials.get("SECRET_ACCESS_KEY")
     40         aws_session_token = ctx_credentials.get("SESSION_TOKEN")

AttributeError: 'str' object has no attribute 'get'

Does line 37 need something like:

ctx_credentials = json.load(prefect.context.get("secrets", {}).get("AWS_CREDENTIALS", {}))

or perhaps a schema validated load like:

from marshmallow import Schema, fields

class AWS_CREDENTIALS_Schema(Schema):
   ACCESS_KEY = fields.Str(required=True)
   SECRET_ACCESS_KEY = fields.Str(required=True)
   SESSION_TOKEN = fields.Str(required=False)

...

schema = AWS_CREDENTIALS_Schema()
ctx_credentials = schema.load(prefect.context.get("secrets", {}).get("AWS_CREDENTIALS", {}))

Hi @Bernard Greyling - I think this should be moved to GitHub since it's a little out of scope of a community thread. I'll open an issue for it now

kk

@Marvin open "AWS context secret manager isn't automatically loading credentials"

seems marvin had some parsing issues there😂

Oof, I'll fix that 🤦