Does task decorator currently support python generators? Trying to see if tasks play nice with yield.
Copy code
@task
def generator(limit: int):
logger = prefect.context.get("logger")
for i in range(limit):
<http://logger.info|logger.info>(i)
yield i*i
k
Kevin Kho
02/07/2022, 5:21 PM
We don’t support generators
h
Heeje Cho
02/07/2022, 5:22 PM
Sounds good, are there plans in Orion to support generators at all?
k
Kevin Kho
02/07/2022, 5:23 PM
That is a good question. I think so because you can use native Python inside a Flow. I am just not sure if you can use it inside a task
h
Heeje Cho
02/07/2022, 5:49 PM
Looks like you can use it inside of a task which will be good enough for now!
Heeje Cho
02/07/2022, 5:49 PM
Copy code
@task
def parse_gen(limit: int):
logger = prefect.context.get("logger")
empty = []
for i in generator(limit):
if i < 100:
<http://logger.info|logger.info>(i)
empty.append(i)
return empty
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.