Preston Marshall
02/06/2020, 10:58 PMChris White
02/06/2020, 11:21 PMPreston Marshall
02/06/2020, 11:29 PMGCSUpload
task, and it seems like it expects the whole file as a string? That seems like it would cause a lot of problems, serializing multi-gigabyte files and sending them over the wire. Am I missing something?Chris White
02/06/2020, 11:30 PMPreston Marshall
02/06/2020, 11:35 PMChris White
02/06/2020, 11:39 PMLocalExecutor
and run your flows in a non-dockerized environment, then all tasks will run in the same process on the same machine so it’s relatively easy to reason about locality in that scenario