01/23/2022, 8:29 PM
hey y'all, do tasks support returning generators? Currently, when I return a generator from a task, the results don't get returned or get picked up in the next task. Couldn't find anything in the docs about it


01/23/2022, 9:19 PM
Hey, could you post a specific example? My basic test works well:
Copy code
from prefect import Flow, task, unmapped

def printl(elem, r):
    print(f"{elem} is produced by {r}")

def get_gen(end):
    return range(end)

with Flow("Test 1") as flow:
    g = get_gen(15), unmapped(g))

if __name__ == "__main__":
My bad, turns out range works but a custom generator fails the receiving task.

Anna Geller

01/23/2022, 9:47 PM
@Ashton can you explain a bit more what type of problem do you try to solve when you return a generator in your task? There might be an alternative approach depending on what you try to do - do you feed it into a map operation?


01/23/2022, 10:23 PM
Thanks for the help! I'm just yielding data from sequential API calls with different parameters. I solved this problem by appending the data into a list and returning it. I was just testing using a generator and noticed that the generator doesn't do anything in a prefect task. Not a huge issue, but was just curious.

Anna Geller

01/23/2022, 10:36 PM
That makes a lot of sense, thanks for sharing your use case!

Kevin Kho

01/24/2022, 3:34 AM
Yes you are right that tasks can’t output generators