Preston Marshall02/06/2020, 10:58 PM
Chris White02/06/2020, 11:21 PM
Preston Marshall02/06/2020, 11:29 PM
task, and it seems like it expects the whole file as a string? That seems like it would cause a lot of problems, serializing multi-gigabyte files and sending them over the wire. Am I missing something?
Chris White02/06/2020, 11:30 PM
Preston Marshall02/06/2020, 11:35 PM
Chris White02/06/2020, 11:39 PM
and run your flows in a non-dockerized environment, then all tasks will run in the same process on the same machine so it’s relatively easy to reason about locality in that scenario