is there a way to get all the subflow from a flow
07/28/2023, 8:47 PM
Oh, the joy of explaining subflows. Subflows are a way to group related tasks within your workflow. They are like mini-flows within a larger flow, and they are useful for a variety of reasons:
1. Observability: Subflows, like any other flow run, have first-class observability within the Prefect UI and Prefect Cloud. You'll see subflow status in the Flow Runs dashboard rather than having to dig down into the tasks within a specific flow run.
2. Conditional flows: If you have a group of tasks that run only under certain conditions, you can group them within a subflow and conditionally run the subflow rather than each task individually.
3. Parameters: Flows have first-class support for parameterization, making it easy to run the same group of tasks in different use cases by simply passing different parameters to the subflow in which they run.
When a subflow starts, it will create a new task runner for tasks within the subflow. When the subflow completes, the task runner is shut down. Subflows will block execution of the parent flow until completion. However, asynchronous subflows can be run in parallel by using AnyIO task groups or asyncio.gather.
Subflows differ from normal flows in that they will resolve any passed task futures into data. This allows data to be passed from the parent flow to the child easily. The relationship between a child and parent flow is tracked.
If you want to get all the subflows from a flow, you would need to traverse the flow's tasks and identify those that are subflows. Unfortunately, there isn't a built-in method to do this, but it's not impossible. Just another delightful challenge for us to solve. Read more here.