I’ve seen subflows mentioned a few different places around here. I know they aren’t supported in a first-class way yet, and was wondering if they are on the roadmap and if so whether there is an expected release date. It would help us decide how much working effort we need to put into workarounds for the short term

Hi @Ben Fogelson, work on subflow design will actually begin this week. It’s a little premature to say when it’ll be ready for release, but I’d expect it’s a matter of weeks (not days or months).

Thanks @Jeremiah! Very excited about that

We are too!

Hi @Jeremiah! Wondering if there are any updates on subflow development timeline

Hi Ben, subflow progress and discussions are being tracked in this issue. However, one of our resulting conclusions when we examined the feature requests was that most requests for subflows were actually requests for grouping tasks during visualization and in the UI. Therefore, one action item is to allow the UI / viz to group up tasks that have the same

tag

, for easy interaction. Another conclusion was that many users were actually just looking for easy ways to add groups of tasks to an existing flow, for which we have encouraged the use of factory functions and possibly

flow.update()

. Lastly, one outstanding item was people who wanted to use subflows to combine multiple “full-scale” flows. For this, we’ve introduced a task that can launch other flows via the Server/Cloud API and are working on a first class “flow-to-flow” dependency structure with in the API. So across these three - adding groups of tasks to a flow; visualizing groups of tasks; and combining flows - it appears that the use cases that motivated subflow feature requests will be met without needing to introduce the complexity of a subflow object. Would those constructs meet your needs as well?

Hi @Jeremiah, thanks for the reply. Took a look at the issue thread, very useful context. Those use cases do cover some of what we want, specifically re visualizations. We also do want the ability to use subflows to combine multiple “full-scale” flows, but for us it would be far more useful if that were part of the prefect core API than the orchestration API. I realize that this gets back to the subflow object and all the complexity there, but I don’t think having flow-to-flow dependency within the server/cloud API would meet our anticipated use case. One particular thing we’d like to do is be able to use something like

parent_flow.replace(old_child_flow, new_child_flow)

to swap out an entire subflow for another one with the same root and terminal tasks.

Got it. We’re definitely looking at ways of advancing primitives for manipulating flows (like

replace

) that would allow users to build up their desired level of complexity