Jeff Kehler07/06/2022, 3:54 AM
a task output, or it doesn’t really help because results are held in memory. I assume you know of mapping but it doesn’t fit your use case either right?
Jeff Kehler07/06/2022, 4:26 AM
Lets pretend that
@task def get_api_data(): items =  for r in stripe.BalanceTransaction.list(limit=100).auto_paging_iter(): items.append(r) return items @task def insert_to_db(records): # logic to insert into db here with Flow("test") as flow: items = get_api_data() insert_to_db(items)
returns 10's of thousands of records. All of this would have to be held in memory before passing it to the insert into db task.