Hi all ! New to Prefect here :i_love_you_hand_sign...
# ask-community
s
Hi all ! New to Prefect here ๐ŸคŸ I'm working on a flow that would send data retrieved using MySQLFetch() to an S3 bucket. What would be the best way to transform the tuple that returned from the MySQLFetch() so that it can be written into an S3 in the appropriate format ? Currently I'm getting this error if I'm sending the results directly to the S3Upload() function :
Copy code
stream = io.BytesIO(data.encode())
AttributeError: 'tuple' object has no attribute 'encode'
k
Maybe try turning it into binary with pickle and then encode?
a
I am brand new to prefect and just looking to learn. But I think how you do this depends on what you want the format of the output file to be. The following is for putting the data in a CSV:
Copy code
import csv, io

with io.StringIO() as fh:
    writer = csv.writer(fh, lineterminator='\n', quotechar='"')
    writer.writerows(data)
    fh.seek(0, 0)
    contents_for_s3upload = fh.read()
Then you can pass
contents_for_s3upload
to
S3Upload.run
.
upvote 1
k
Thatโ€™s a good suggestion!
a
Glad to be here learning!
s
Awesome thanks alot!
I actually found a way to upload with
S3Results()
but now I have to make sure that I can actually read the results back in the appropriate format!
If not, I'll definitely use your approach
Thanks alot for the help ๐Ÿ™‚
๐Ÿ‘ 1
k
You can make your own serializer and add it to the result
๐Ÿ‘ 1