https://prefect.io logo
Title
n

Nelson Griffiths

01/14/2023, 6:36 PM
Am I correct in understanding that currently there is no way to execute subflows as if they were tasks with a task runner? For example, if I want to run three subflows in parallel with Ray that isn’t currently possible right?
βœ… 1
z

Zanie

01/14/2023, 7:33 PM
I'm very hesitant to allow subflows to be submitted to external task runners because then if they have a task runner of their own we can end up in a complex nested situation.
:gratitude-thank-you: 1
n

Nelson Griffiths

01/14/2023, 8:00 PM
Yeah that makes sense. It just feels like there should be som way to run them in parallel
z

Zanie

01/14/2023, 9:03 PM
Definitely, we can do it in local threads it's on the roadmap after some other concurrency improvements :)
n

Nelson Griffiths

01/14/2023, 9:26 PM
Okay great! Looking forward to it!
@Zanie would you recommend just using tasks right now if I need things parallelized within a single flow?
z

Zanie

01/14/2023, 9:57 PM
Generally yeah, or subdeployments or async subfows
πŸ‘ 4
a

Anna Geller

01/14/2023, 9:59 PM
Nelson, parallel subdeployments could look like this:
from prefect.deployments import run_deployment

for _ in range(1, 100):
    run_deployment(name="healthcheck/dev", timeout=0)
πŸ‘ 1
n

Nelson Griffiths

01/14/2023, 10:11 PM
@Anna Geller how can I wait on a deployment to finish before continuing with the flow?
a

Anna Geller

01/15/2023, 2:25 AM
remove the timeout=0 πŸ™‚
πŸ™Œ 1
I thought the intention was to fire a bunch of runs from a deployment in parallel so I set it to 0 here for fire and forget behavior but the default behavior (no timeout) is to wait indefinitely until the runs completes
n

Nelson Griffiths

01/15/2023, 2:29 AM
Thank you!
πŸ™Œ 1