Hi - looking to pass AWS credentials to a Kubernetes agent at runtime so as to interface with the pynamoDB python library. I’m getting the following error when I attempt to scan a table in DynamoDB:

Failed to scan table: Unable to locate credentials

I have

AWS_CREDENTIALS

stored in Prefect cloud as a Secret, and I’ve tried passing credentials as envs passed to KuberenetesRun:

env={
        "ACCESS_KEY": Secret("AWS_CREDENTIALS").get().get("ACCESS_KEY"),
        "SECRET_ACCESS_KEY": Secret("AWS_CREDENTIALS").get().get("SECRET_ACCESS_KEY")
    }

Returns the same error. What’s the best / preferred approach to authenticate a Kubernetes agent against AWS?

There are certainly many ways to go about it. The best approach would probably be IAM roles for service accounts. But as a quick solution, perhaps you could try attaching a default boto3 session within your task, i.e. moving the Secret().get() into the boto3.setup_default_session:

I should’ve mentioned this before - I can authenticate with boto3 and access DynamoDB no problem (we’re refactoring and everything uses the pynamo library today so I’m trying to lift and shift with minimal refactoring), but I will look at adding an IAM role for a service account specific to Prefect. Thank you!