Hi community, if we do not use flow of flows framework, is there any downside? Basically from my understanding flow of flows is to use one single flow (orchestrator) to manage other flows (dependency, schedule etc.) Instead of defining those relationship in each flow's script?

Hi ! Your understanding is very correct. It is a way to orchestrate multiple flows that need to run in order. Another common use case is to separate parts of a big flow so that they can run on their own if needed e.g. an end to end machine learning flow that orchestrates a data processing flow, a feature engineering, a training flow, etc. If you don't need to do this, you really don't need to and it actually makes code slightly more readable in my opinion.

Chu, it depends on your use case - flow of flows is particularly helpful for larger data warehousing projects where you may run hundreds of jobs to populate and transform your data warehouse tables and doing it all in a single flow without something like flow-of-flows may end up like this (which is nightmare to maintain)

I expect I'll have nightmares from this tonight. Thanks Anna 😛

you're very welcome 😄

but why

Thanks for your vivid explanation, successfully sell me the idea 😉 . One last question, if going with the nightmare like that, we need to define flow dependency when we created each flow, right? Like I already defined 5 flows, they have some dependencies. In flow of flows, I can just create a high level flow to mange those 5 flows. If nor using flow of flow, do I need to set those dependency in each of flows? If not, how can i do that?

Maybe we misunderstood your initial question. If your flows have some dependencies between them like the order in which they need to run, I'd say flow of flows is the best approach in order to manage it properly.