Is this still the best way to run subflows in para...
# ask-communityk
kasteph
05/22/2023, 3:15 PMIs this still the best way to run subflows in parallel? https://discourse.prefect.io/t/how-can-i-run-multiple-subflows-or-child-flows-in-parallel/96
d
Deceivious
05/22/2023, 3:23 PMYea
k
kasteph
05/22/2023, 3:29 PMwhat if subflows have to be dynamically created as in a for-loop?
kasteph
05/22/2023, 3:30 PMwould it work just as well to do like so:
Copy code
parallel_subflows = []
for s in stuff:
subflow = subflow_func(s)
parallel_subflows.append(subflow_func())
await asyncio.gather(*parallel_subflows)d
Deceivious
05/22/2023, 3:36 PMDeceivious
05/22/2023, 3:36 PMIt's the same
k
kasteph
05/22/2023, 4:22 PMNeither the Prefect example nor mine use a list comprehension
d
Deceivious
05/22/2023, 4:33 PMAh yes but you can pass dynamic args and it'll work
k
kasteph
05/22/2023, 4:34 PMthanks, testing it now!
n
Nate
05/22/2023, 7:53 PMin case you want your subflows to be a standalone deployment @kasteph, i'll leave this here
k
kasteph
05/22/2023, 7:53 PMwow, lifesaver 🫶
🦜 1
kasteph
05/22/2023, 8:35 PM@Nate could asyncio run the apply method of the deployment instead (assuming using
Deployment.build_from_flow Copy code
asyncio.run(get_total_pokemon_weight_deployment.apply())n
Nate
05/22/2023, 8:45 PMit depends on whether the context you're calling from is async or sync. if you're in an sync context, you should just be able to call
get_total_pokemon_weight_deployment.apply()awaitNate
05/22/2023, 8:45 PMshouldnt have to use
asyncio.runk
kasteph
05/22/2023, 10:13 PMworked nicely, thanks 🙂
n
Nate
05/22/2023, 10:15 PM👍
3 Views