https://prefect.io logo
Title
c

Choenden Kyirong

04/06/2023, 7:15 AM
is it possible to run 2 tasks from 1 task? For example:
--> task_b --
        |            |
task_a -             ----> task_d
        |            |
         --> task_c --
If so, will
task_d
wait for both
task_b
and
task_c
to finish before executing? Any guidance on this would be greatly appreciated- Thanks!
โœ… 1
m

Matt Fysh

04/06/2023, 11:30 AM
tasks can't call other tasks, I think you might be talking about flows/subflows? im pretty new to prefect but I think what you want to do is make your flow async, and then
@flow
async def my_flow():
  future1 = task_b()
  future2 = task_c()
  result1 = await future1
  result2 = await future2
  task_d(result1, result2)
n

Nathan Low

04/06/2023, 12:41 PM
You might want to use wait_for: https://docs.prefect.io/concepts/flows/#return-a-future
@flow
def always_succeeds_flow():
    x = always_fails_task.submit().result(raise_on_failure=False)
    y = always_succeeds_task.submit(wait_for=[x])
    return y
๐Ÿ‘ 1
You'll want a flow, that runs one task, has two tasks that wait for the 1st task, and a 4th task that waits for task 2 and 3.
๐Ÿ™Œ 1
c

Choenden Kyirong

04/06/2023, 8:56 PM
hmm, okay. Iโ€™ll look into these. Thanks @Nathan Low @Matt Fysh
@Nathan Low Would this be something to similar to airflows:
first_task >> [second_task, third_task]
third_task << fourth_task

or 

first_task.set_downstream(second_task, third_task)
third_task.set_upstream(fourth_task)
which produces a flow like this:
I think youโ€™re right @Nathan Low.
wait_for
might do it:
@flow
def my_flow():
    x = task_a()
    future2 = task_b(x).submit()
    future3 = task_c(x).submit()

    z = task_d(wait_for=[future2, future3])
m

Matt Fysh

04/07/2023, 12:26 AM
if you don't need to pass data from tasks B &C into task D, then wait_for is what you're looking for: https://docs.prefect.io/latest/concepts/tasks/#wait-for
๐Ÿ‘ 1
๐Ÿ™Œ 1
c

Choenden Kyirong

04/07/2023, 12:31 AM
okay, great. Thanks @Matt Fysh