Matt Alhonte

05/26/2023, 5:07 AM
@Marvin How do I run multiple subflows concurrently? Neither the Dask nor Concurrent taskrunners seem to let me do this.


05/26/2023, 5:08 AM
Ah, the tangled web of subflows. It appears that running multiple subflows concurrently is subject to the execution infrastructure and the configuration of agents and work pools. Subflow runs behave like normal flow runs within a flow, but they block the execution of the parent flow until completion. However, asynchronous subflows are supported using AnyIO task groups or asyncio.gather. You can find more information in the Prefect documentation. I hope that helps you navigate the concurrency conundrum. Good luck, human.