Nesting multiple asynchronous flows in Prefect can...
# ask-marvin
m
Nesting multiple asynchronous flows in Prefect can be managed using the concept of subflows or nested flows. Here's how it works, based on both Prefect 2.x and 3.x: ### Prefect 2.x In Prefect 2.x, you can create subflows by calling a flow function inside another flow. The parent flow is the main flow, while the child or "subflow" is executed within it. Subflows behave like regular flow runs and have their own task runners. Importantly, subflows block the parent flow's execution until completion, but with asynchronous subflows, you can run them concurrently using
asyncio.gather
. For more on this, you can check the Prefect 2.x documentation on nested flows. ### Prefect 3.x In Prefect 3.x, nested flows offer several benefits, such as using different task runners and enabling conditional execution of grouped tasks. They provide first-class observability, meaning you can track the status of nested flows easily within the Prefect UI. Nested flows also support asynchronous execution, allowing them to be awaited or run concurrently, adhering to async Python standards. For a deeper dive into creating asynchronous nested flows in Prefect 3.x, review the Prefect 3.x documentation on writing flows. In essence, whether you're using Prefect 2.x or 3.x, the key to nesting asynchronous flows lies in leveraging subflows with async capabilities and managing their execution so they can handle concurrent workloads effectively. Ah, the joys of managing asynchronous tasks—always making you feel like a juggler who needs just one more hand!
h
Ok so for prefect 3 I would use asyncio.gather? Can you give a code example please