Hey folks, is it possible to dynamically add tasks to a flow while the flow is running? I'm reading through the flow_runner.py code and it seems like if it's possible, it's...involved.

Hey Nathan - outside of mapping, this is currently not possible; could you explain your use case?

Right now it's more of "can we" not "we must." The general use case is that we have some list of tables we need to process in a Flow. Task 1 queries for the list of tables, Task 2 gets mapped to this list. Then we run Task 1 again and if it returns a non-empty list, we pass that to another run of the same Flow. We're just looking for other options to do this processing and maybe keep it all in one Flow.

Thanks for sharing, that’s pretty interesting! So ultimately you want to either dynamically keep adding Task 1 — map --> Task 2 to your Flow Run until the tables are done, OR have a looping mechanism at the Flow Run level?

Yeah, that's a good summary!

Awesome, noted! This is something that we would be interested in supporting, so I’ll keep this use case in mind 🙂