Got an Orion question: Will I be able to map subfl...
# ask-communityt
Tim Enders
11/10/2021, 6:24 PMGot an Orion question: Will I be able to map subflows across a Dask cluster?
Tim Enders
11/10/2021, 6:27 PMMaybe async flows is more what I am looking for? I have a pattern where I get a list of work, and then I would like to map across the list and run the flows in parallel
z
Zanie
11/10/2021, 6:43 PM
Hey Tim! Async subflows can indeed run in parallel.
Zanie
11/10/2021, 6:44 PM
Child flow runs are run in the main thread of the parent flow run. They're not submitted to the executor like tasks.
t
Tim Enders
11/10/2021, 6:44 PMDo you guys have documentation on async flows yet?
z
Zanie
11/10/2021, 6:48 PM
Let me look 😄
Zanie
11/10/2021, 6:49 PM
Zanie
11/10/2021, 6:50 PM
There's not a lot there yet, but basically we just return coroutines if you've got an async def function.
t
Tim Enders
11/10/2021, 6:50 PMI missed that there were tutorials already! great
z
Zanie
11/10/2021, 6:50 PM
You can asyncio.gather a bunch of async subflow runs to get them to run in paralllel.
Zanie
11/10/2021, 6:50 PM
We'll be introducing a
.mapt
Tim Enders
11/10/2021, 6:51 PMOK, awesome. Orion has me so excited
marvin 1
z
Zanie
11/10/2021, 6:54 PM
Me too 🙂 let me know if you run into any issues with async.
Zanie
11/10/2021, 6:54 PM
In theory, the async implementation is much simpler because we don't have to wrap sync code.
t
Tim Enders
11/10/2021, 6:56 PMWill do. Right now I am seeing if it is far enough along for me to start trying out. I don't want to just run into basic problems related to my workflows that you all are simply working on and haven't released. I think I will start trying some stuff out next week.
5 Views