Hi there. I am trying to understand the default secrets. Are the default AWS credential secrets only for use with tasks from the task library or are they supposed to be passed to all instantiations of the boto3 client inside of a prefect task? I defined

PREFECT__CONTEXT__SECRETS__AWS_CREDENTIALS

in a

.env

file that I am loading with

dotenv.load_dotenv()

The env variables are being passed to the Python script OK. Then inside a task, I create an s3 client that is used in a module I created (seudo code):

@tasks
def mytask:
    api_client = MyPrivateClient(s3_client=boto3.client("s3"))
    api_client.execute()

Then I get

NoCredentialsError

After reading https://docs.prefect.io/core/concepts/secrets.html#default-secrets I thought the credentials would be exposed as environment variables for the client to use. Should I be getting them explicitly from the context or using the default env vars that boto3 will look for?

Hi Pedro, it would be bad practice if Prefect manipulated your local environment in any way that you didn’t explicitly configure yourself; only pieces of the builtin Prefect library that interact with AWS will use the Prefect AWS CREDENTIALS secret - if you write a fully custom task (like you are doing here) you need to configure authorization yourself. In this case, if you want to use prefect built-ins so that your secret is used, you might consider creating a boto3 client as follows:

from prefect.utilities.aws import get_boto_client

s3_client = get_boto_client('s3')

The above code should create a client that uses your Prefect secret as you expect

Hi Chris. This is great and would work fine for my use case. Is it a bad pattern to access the prefect secrets from the context? I haven't looked into it but I imagine it's possible.

awesome! Accessing secrets directly from context yourself is also a valid way to do this