https://prefect.io logo
Powered by
Title
m

Mark NS

01/23/2023, 2:18 PM
Hi folks. I'm trying to use the S3 block to upload a file, however the resulting 
content-type
on the uploaded file is always 
binary/octet-stream
s3bucket: S3 = S3.load("default")
    with open(local_path, "rb") as f:
        s3bucket.write_path(to_path, f.read())
Does anyone have a pointer on how to set the metadata on upload? @Marvin ?
✅ 1
r

Ryan Peden

01/23/2023, 4:32 PM
The built-in 
S3
block has limits since it's built on top of 
fsspec
. In the `prefect-aws` collection, we have an S3Bucket block with an upload_from_file_object method I think you could use here. It has a 
**kwargs
param you could use to pass metadata to the underlying boto3 library via ExtraArgs.
m

Mark NS

01/23/2023, 5:04 PM
Awesome, thanks for the tip @Ryan Peden
This works well. Thanks again. If anyone runs into this in future, here's some code: 
def upload_html(s3bucket: S3Bucket, html: str):
    with BytesIO() as f:
        f.write(html.encode())
        f.seek(0)
        s3bucket.upload_from_file_object(f, "analysis/index.html", ExtraArgs={
            "CacheControl": "no-cache",
            "ContentType": "text/html"
        })
👍🏻 1
r

Ryan Peden

01/24/2023, 12:49 PM
You're welcome, Mark! I'm happy it worked well. And thanks for sharing your code!
#prefect-communityJoin Slack
Powered by