@Marvin in Prefect 3 how do I run multiple subflows concurrently?

In Prefect 3.x, to run multiple subflows concurrently, you can utilize the

flow

decorator to define your subflows and use the

concurrent.futures

module to handle concurrent execution. Here's a basic outline of how you might achieve this: 1. Define Subflows: Use the

@flow

decorator to define each subflow. 2. Concurrent Execution: Utilize Python's

concurrent.futures.ThreadPoolExecutor

or

ProcessPoolExecutor

to run these subflows concurrently. Here's a simple example:

python
from prefect import flow
from concurrent.futures import ThreadPoolExecutor

@flow
def subflow1():
    # Subflow logic here
    pass

@flow
def subflow2():
    # Subflow logic here
    pass

@flow
def main_flow():
    with ThreadPoolExecutor() as executor:
        futures = [
            executor.submit(subflow1),
            executor.submit(subflow2)
        ]
        for future in futures:
            future.result()  # Wait for each subflow to complete

if __name__ == "__main__":
    main_flow()

This approach leverages Python's built-in concurrency tools to run subflows in parallel. For more details, you can refer to the Prefect documentation and explore the relevant sections on flow and concurrency.

fwiw i would think this is fine for sync flows, but for async flows id just gather them like normal python functions