So I'm digging into the prefect repo and see that ...
# ask-community
c
So I'm digging into the prefect repo and see that AWS credentials are handed off as a credential dict to a MagicMock() function. The reason I'm looking is because passing CodeCommit(secrets=) does not work outright whereas S3(secrets=) does work, and our flows run remotely. While trying to pass in client_options={'credentials':['AWS_Credentials']} yields "TypeError("get_boto_client() got multiple values for keyword argument 'credentials'",)". This makes sense as this seems to be how prefect calls boto:
Copy code
@property
    def _boto3_client(self):  # type: ignore
        from prefect.utilities.aws import get_boto_client

        kwargs = self.client_options or {}
        return get_boto_client(resource="codecommit", credentials=None, **kwargs)
and then the get_boto_client func:
Copy code
def get_boto_client(
    resource: str,
    credentials: Optional[dict] = None,
    region_name: Optional[str] = None,
    profile_name: Optional[str] = None,
    **kwargs: Any
) -> "botocore.client.BaseClient":
but it seems it holds onto None and doesn't let anything else pass. Is there anyone else here that has successfully connected to CodeCommit through Prefect?
z
Hi @Charles Liu -- The credentials are set to
None
because they rely on the
get_boto_client
credential behavior
Copy code
if credentials:
        aws_access_key = credentials["ACCESS_KEY"]
        aws_secret_access_key = credentials["SECRET_ACCESS_KEY"]
        aws_session_token = credentials.get("SESSION_TOKEN")
    else:
        ctx_credentials = prefect.context.get("secrets", {}).get("AWS_CREDENTIALS", {})
        aws_access_key = ctx_credentials.get("ACCESS_KEY")
        aws_secret_access_key = ctx_credentials.get("SECRET_ACCESS_KEY")
        aws_session_token = ctx_credentials.get("SESSION_TOKEN")
1
It seems like
secrets=...
should work the same as in S3
The only difference between the boto client property in each of these storage options is that the CodeCommit storage asks for the
codecommit
resource.
Perhaps your secret credentials are lacking the proper permissions?
c
Ah okay gotcha. They shouldn't, I'm more or less an admin account and my key/secret works with S3 storage, being passed in with a secret set on the cloud front end.
To clarify, it is the same account I use for git control in my IDE and we use CodeCommit there as well.
verdict: repo="" does not like URLs and it's literally just the repo name DOH!
Thanks so much to the Prefect team for all the help so far! We've made some great strides on our side re: implementing Prefect.
❤️ 1