Hi everyone, I’m trying to run subflows in parallel. These subflows in turn call other flows. This is somewhat of a setup I have,

import asyncio
from prefect import flow

@flow
async def subflow_1():
    print("Subflow 1 started!")
    some_other_flow()
    # post result ops

@flow
async def subflow_2():
    print("Subflow 2 started!")
    some_other_flow_2()
    # post result ops


@flow
async def subflow_3():
    print("Subflow 3 started!")
    some_other_flow_3()
    # post result ops


@flow
async def subflow_4():
    print("Subflow 4 started!")
   some_other_flow_4()
    # post result ops


@flow
async def main_flow():
    parallel_subflows = [subflow_1(), subflow_2(), subflow_3(), subflow_4()]
    await asyncio.gather(*parallel_subflows)

if __name__ == "__main__":
    main_flow_state = asyncio.run(main_flow())

Now when I deploy this (or run locally), flow runs are created for the 4 subflows, but that’s about it, thereafter it’s all sequential, nothing gets processed parallely. For Eg: subflow_2 run proceeds only after subflow_3(in turn some_other_flow_3) have finished. So in the end it makes no difference to calling each of these subflows sequentially instead of using asycio Am I missing something else here? Kindly advice

There is a difference between concurrent and parallel execution. Asyncio allows for concurrent execution, where an event loop in one thread handles switching of context between tasks when awaited. It is most useful for IO bound tasks, where while a task is waiting for a server response for example another can start it's execution concurrently

ah, sorry I should’ve probably written concurrent, I’m sort of looking for similar behaviour to how ConcurrentTaskRunner works in case of tasks. These flows which are within the subflows make calls to microservices etc, so I presume that other flows should be running concurrently when one of the flows are waiting? M sorry if my understanding sounds limited, m working on concurrency for the first time

maybe try submitting the tasks inside the subflows (or make them awaitable) ? because as it is the tasks are probably blocking execution