Hey I am a little confused on how to incorporate mapping to uploading to S3. I am using the first task to get all of the files. 2nd step to modify each file to a new json in memory. Finally, the 3rd task I need to upload those new files. Should I save the files locally before uploading? or is there a way to upload them in the same step of the mapping?

Hi @Kyle Pierce - it sounds like you could either upload them in the same step as you modify them OR, since you're converting them to JSON anyway, you can return each JSON string to an upload task instead. So you could get / read files to json => upload them

@nicholas thanks for response. I try putting a boto3 client s3 put_object into the 2nd task, but I get an error everytime. "Could not infer an active Flow context."

@Kyle Pierce can you share a fuller version of your code? It sounds like you're trying to call a task from outside a flow context.