Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

hey! can I use prefect.tasks.aws.s3.S3Upload to upload a parquet file?

Hi <@U026F9PBGDP>, actually seems like this task can’t do that. It seems this was meant for string/json data. What are you using to produce the parquet? Pandas/Spark/Dask?

Because my suggestion might be to use s3fs to write out that parquet so you can just do `<http://dd.to|dd.to>_csv("s3://...")` or something like that. s3fs can make this easier.