Colton
06/04/2024, 10:27 PMMarvin
06/04/2024, 10:27 PMSecretStr
for the secret access keys to ensure they are not exposed.
python
from typing import Optional
from prefect.blocks.core import Block
from pydantic import SecretStr
class AWSCredentials(Block):
aws_access_key_id: Optional[str] = None
aws_secret_access_key: Optional[SecretStr] = None
aws_session_token: Optional[str] = None
profile_name: Optional[str] = None
region_name: Optional[str] = None
def get_boto3_session(self):
import boto3
return boto3.Session(
aws_access_key_id=self.aws_access_key_id,
aws_secret_access_key=self.aws_secret_access_key.get_secret_value(), # Extract the secret value
aws_session_token=self.aws_session_token,
profile_name=self.profile_name,
region_name=self.region_name
)
### Using the AWS Credentials Block with boto3
Here’s how you can use this block to get an authenticated session with `boto3`:
python
aws_credentials_block = AWSCredentials.load("my-aws-credentials-block")
boto3_session = aws_credentials_block.get_boto3_session()
athena_client = boto3_session.client("athena")
# Now you can use athena_client to interact with Amazon Athena
### Documentation Links
For more details, you can refer to:
- Prefect Blocks Concept
- Prefect Data Movement Guide
### GitHub Issues
There were relevant GitHub issues discussing similar topics:
1. Make credentials optional for S3Bucket block
2. push_project_to_s3 not working with credentials in a Prefect block
These issues provide additional insights and proposals for improvements regarding credential handling in Prefect blocks.Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by