Hi all, given that during flow_run, task_A will return a list from a sql query downstream to task_B. task_B will then start at index[0] and send that value downstream to task_C. Now, x minutes later, flow_run will run again, and task_A will return the same list, but task_B will now return index[1] instead of index[0] and send the value of index[1] to task_C and so on and so forth. What would be the best approach for this, where I want a given flow, between scheduled runs, to remember its location in an iterable?

Hey @Dominic Pham — this sounds like a great use for the KV store https://docs.prefect.io/orchestration/concepts/kv_store.html

Thanks for the response, I'll take a look into this!

I think you just need to persist the counter in a file and then read and write from it

You could also update a parameter default for the flow at the end of each flow run, but that's a hack