For instance, when processing a CSV, can you yield per row?
j
Jeremiah
11/11/2019, 4:22 PM
Hi @Jason, at this time Prefect doesn’t support the generator model; each task must completely finish before downstream tasks run. However, you can use the
Task.map()
operation to map a downstream task over each row of a returned CSV.
c
Chris White
11/11/2019, 4:22 PM
Hi Jason - returning generators is not currently supported. Another reason for this is that generators do not work well in distributed modes