Matheus Rocha
04/18/2023, 5:30 PMA >> [B, C]
[B, C] >> D
E
Nate
04/18/2023, 9:20 PM>>
creates a state dependency between some tasks.
In prefect, you can have either/both state dependencies or/and data dependencies. To create your example in prefect with only state dependencies
from prefect import flow, task
# implement tasks A-E, assume no data is passed between them
@flow
def my_flow():
task_a_result = task_a()
task_b_result = task_b(wait_for=[task_a_result])
task_c_result = task_c(wait_for=[task_a_result])
task_d_result = task_d(wait_for=[task_b_result, task_c_result])
task_e(wait_for=[task_d_result])
or if hypothetically you wanted to pass data between these tasks
A >> [B, C]
[B, C] >> D
you could change that to create implicit data dependencies (and state! since the result is associated with the state)
# implement tasks A-E, assume B and C accept A's result as a kwarg, D accepts B and C's result as a kwarg
# more just like regular python
@flow
def my_flow():
task_a_result = task_a()
task_b_result = task_b(task_a_result)
task_c_result = task_c(task_a_result)
task_d_result = task_d(task_b_result, task_c_result)
task_e(wait_for=[task_d_result]) # waits for D's result to materialize before running