<@ULVA73B9P> Can you show me how I would use `pref...
# ask-marvin
d
@Marvin Can you show me how I would use
prefect-aws
to pull and push data from an s3 bucket?
m
thought for 162.7 seconds
Absolutely. Here’s a minimal, working way to push and pull data to S3 with prefect-aws in Prefect 3.x. Nothing flashy, just effective. Install the AWS extra
Copy code
pip install "prefect[aws]"
Create or load AWS credentials (Block) - You can save a reusable block in code or create it in the UI. Save once in code:
Copy code
from prefect_aws.credentials import AwsCredentials

AwsCredentials(
    aws_access_key_id="YOUR_KEY",
    aws_secret_access_key="YOUR_SECRET",
    region_name="us-east-1",
).save("aws-creds", overwrite=True)
Or rely on IAM role/env by just doing
AwsCredentials()
at runtime. Push (upload) bytes to S3
Copy code
from prefect import flow
from prefect_aws.credentials import AwsCredentials
from prefect_aws.s3 import upload_to_bucket

@flow
def push_example():
    aws = AwsCredentials.load("aws-creds")  # or AwsCredentials()
    data = b"hello from prefect-aws"
    key = "examples/hello.txt"
    etag = upload_to_bucket(
        data=data,
        bucket="my-bucket",
        key=key,
        aws_credentials=aws,
    )
    return etag

if __name__ == "__main__":
    push_example()
Pull (download) bytes from S3
Copy code
from prefect import flow
from prefect_aws.credentials import AwsCredentials
from prefect_aws.s3 import download_from_bucket

@flow
def pull_example():
    aws = AwsCredentials.load("aws-creds")  # or AwsCredentials()
    key = "examples/hello.txt"
    content = download_from_bucket(
        bucket="my-bucket",
        key=key,
        aws_credentials=aws,
    )
    print(content.decode())
    return content

if __name__ == "__main__":
    pull_example()
Async versions (if you insist on speed)
Copy code
from prefect import flow
from prefect_aws.credentials import AwsCredentials
from prefect_aws.s3 import aupload_to_bucket, adownload_from_bucket

@flow
async def async_example():
    aws = AwsCredentials.load("aws-creds")
    await aupload_to_bucket(data=b"hi", bucket="my-bucket", key="k.txt", aws_credentials=aws)
    data = await adownload_from_bucket(bucket="my-bucket", key="k.txt", aws_credentials=aws)
    return data
Notes - The upload task returns a string (e.g., ETag). The download task returns bytes. - For path-like/file-style operations (e.g., read/write files rather than raw bytes), the S3Bucket block provides higher-level I/O. See docs below. Docs and references - prefect-aws S3 docs - Prefect docs - Blocks catalog - AwsCredentials block If you tell me whether you want file paths vs raw bytes, and how you’re handling credentials (IAM role vs keys), I can tailor this further. I live to serve. Well, “live” is generous.