Hi all ! New to Prefect here 🤟 I'm working on a flow that would send data retrieved using MySQLFetch() to an S3 bucket. What would be the best way to transform the tuple that returned from the MySQLFetch() so that it can be written into an S3 in the appropriate format ? Currently I'm getting this error if I'm sending the results directly to the S3Upload() function :

stream = io.BytesIO(data.encode())
AttributeError: 'tuple' object has no attribute 'encode'

Maybe try turning it into binary with pickle and then encode?

I am brand new to prefect and just looking to learn. But I think how you do this depends on what you want the format of the output file to be. The following is for putting the data in a CSV:

import csv, io

with io.StringIO() as fh:
    writer = csv.writer(fh, lineterminator='\n', quotechar='"')
    writer.writerows(data)
    fh.seek(0, 0)
    contents_for_s3upload = fh.read()

Then you can pass

contents_for_s3upload

to

S3Upload.run

.

That’s a good suggestion!

Glad to be here learning!

Awesome thanks alot!

You can make your own serializer and add it to the result