Hi all, we are evaluating Prefect and were wondering if it is a good idea to initiate flows within flows

Sure, we use that pattern often. One example is a "syncer" flow which tracks a source of files. Any time there are new files to ingest it kicks off a separate "copy" flow to ingest that set of files.

Hello and welcome ! 👋 As far as I know, it is a common pattern that is even pretty well documented : flow of flow. The usual pattern is having atomic flows that you sometimes want to run by themselves and a "meta" flow that runs those flows in order.

Ah nice, thanks both, I think that answers our question 🙂

Also I believe Orion will allow you to run subflows in the same way you run tasks, making this pattern quite a bit easier to setup and debug.

I haven't come across Orion yet in the docs

Be careful, it is still offered as a technical preview and is not yet deemed production ready by the Prefect team. From what was announced, we can expect to get out of preview in the coming months but nothing more precise as of yet. Depending on how soon you need to use it, it could be a good idea to stick to Prefect.

we are in the evaluation phase now, so good to have a look where things are going anyway

Definitely agree with this ! Just wanted to make sure you wouldn't go full production in the next weeks.

hehe yeah I think the Prefect team will be more quick than our product 😉 thanks for the headsup anyway!

@Sylvain Hazard and @Brian Phillips explained it so well 💯 but if you want to read more about the flow of flows orchestration pattern, we have an entire category in Discourse devoted to that - check this out here: https://discourse.prefect.io/tag/flow-of-flows

Plan is for Orion to graduate from tech preview to beta this month!