For instance, when processing a CSV, can you yield per row?
j
Jeremiah
11/11/2019, 4:22 PM
Hi @Jason, at this time Prefect doesn’t support the generator model; each task must completely finish before downstream tasks run. However, you can use the
Task.map()
operation to map a downstream task over each row of a returned CSV.
c
Chris White
11/11/2019, 4:22 PM
Hi Jason - returning generators is not currently supported. Another reason for this is that generators do not work well in distributed modes
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.