Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi All, I would like to write a pandas dataframe as csv to s3 in prefect, shouldn't this work?
```# upload to s3
write_to_s3 = S3Upload(
    bucket=s3_bucket,
    boto_kwargs=dict(
        endpoint_url=os.getenv("s3_endpoint"),
        aws_access_key_id=os.getenv("s3_access_key"),
        aws_secret_access_key=os.getenv("s3_secret_key")
    )
)

output = write_to_s3(results.to_csv(index=False), key=file_name)```

looking at the code, I see that it is using bytesIO, so I had to encode as below and it works !
```write_to_s3(results.to_csv(index=False).encode(), key=file_name)```

Yeah the task as it is only accepts bytes

<@U02JPHYGCFN> as much as I like our task library, for loading dataframes to S3, <https://aws-data-wrangler.readthedocs.io/en/stable/tutorials/003%20-%20Amazon%20S3.html|awswrangler> has just way better functionality. You can upload your Pandas dataframe as a compressed parquet file in a single command:
```from prefect import task
import awswrangler as wr

@task
def upload_df(df, path):
    wr.s3.to_csv(df1, path, index=False)```