Sidney Fox

    Sidney Fox

    8 months ago
    Hi - looking to pass AWS credentials to a Kubernetes agent at runtime so as to interface with the pynamoDB python library. I’m getting the following error when I attempt to scan a table in DynamoDB:
    Failed to scan table: Unable to locate credentials
    I have
    AWS_CREDENTIALS
    stored in Prefect cloud as a Secret, and I’ve tried passing credentials as envs passed to KuberenetesRun:
    env={
            "ACCESS_KEY": Secret("AWS_CREDENTIALS").get().get("ACCESS_KEY"),
            "SECRET_ACCESS_KEY": Secret("AWS_CREDENTIALS").get().get("SECRET_ACCESS_KEY")
        }
    Returns the same error. What’s the best / preferred approach to authenticate a Kubernetes agent against AWS?
    Anna Geller

    Anna Geller

    8 months ago
    There are certainly many ways to go about it. The best approach would probably be IAM roles for service accounts. But as a quick solution, perhaps you could try attaching a default boto3 session within your task, i.e. moving the Secret().get() into the boto3.setup_default_session:
    Sidney Fox

    Sidney Fox

    8 months ago
    I should’ve mentioned this before - I can authenticate with boto3 and access DynamoDB no problem (we’re refactoring and everything uses the pynamo library today so I’m trying to lift and shift with minimal refactoring), but I will look at adding an IAM role for a service account specific to Prefect. Thank you!