https://prefect.io
Join Slack
is it possible to run 2 tasks from 1 task? For example: ``` --> task_b -- | ...
# ask-community
c
is it possible to run 2 tasks from 1 task? For example:
Copy code
--> task_b --
        |            |
task_a -             ----> task_d
        |            |
         --> task_c --
If so, will 
task_d
wait for both 
task_b
and 
task_c
to finish before executing? Any guidance on this would be greatly appreciated- Thanks!
✅ 1
m
tasks can't call other tasks, I think you might be talking about flows/subflows? im pretty new to prefect but I think what you want to do is make your flow async, and then
Copy code
@flow
async def my_flow():
  future1 = task_b()
  future2 = task_c()
  result1 = await future1
  result2 = await future2
  task_d(result1, result2)
n
You might want to use wait_for: https://docs.prefect.io/concepts/flows/#return-a-future
Copy code
@flow
def always_succeeds_flow():
    x = always_fails_task.submit().result(raise_on_failure=False)
    y = always_succeeds_task.submit(wait_for=[x])
    return y
👍 1
You'll want a flow, that runs one task, has two tasks that wait for the 1st task, and a 4th task that waits for task 2 and 3.
🙌 1
c
hmm, okay. I’ll look into these. Thanks @Nathan Low @Matt Fysh
@Nathan Low Would this be something to similar to airflows:
Copy code
first_task >> [second_task, third_task]
third_task << fourth_task

or 

first_task.set_downstream(second_task, third_task)
third_task.set_upstream(fourth_task)
which produces a flow like this:
I think you’re right @Nathan Low. 
wait_for
might do it:
Copy code
@flow
def my_flow():
    x = task_a()
    future2 = task_b(x).submit()
    future3 = task_c(x).submit()

    z = task_d(wait_for=[future2, future3])
m
if you don't need to pass data from tasks B &C into task D, then wait_for is what you're looking for: https://docs.prefect.io/latest/concepts/tasks/#wait-for
👍 1
🙌 1
c
okay, great. Thanks @Matt Fysh
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by