Hi, I want to ask if there is any way to "connect" two tasks together (so that in the radar UI it is clear that one tasks follows the other), other than task A having as input an output of task B?
a
Anna Geller
01/16/2023, 6:51 PM
you need wait_for
example:
Copy code
from prefect import task, flow, allow_failure
import random
@task
def task_1():
print("task 1 succeeded")
@task
def task_2():
print("task 2 succeeded")
@task
def task_3():
print("task 3 succeeded")
@task
def task_4():
print("This task often fails")
if random.random() > 0.5:
raise ValueError("Non-deterministic error has occured.")
else:
print("task 4 succeeded")
@task
def task_5():
print("task 5 succeeded")
@task
def clean_up_task():
print("Cleaning up ๐งน")
@flow(log_prints=True, name="Cleanup task may not get executed")
def main():
# 1, 2, 3 can run concurrently
one = task_1.submit()
two = task_2.submit()
three = task_3.submit()
four = task_4.submit(wait_for=[one, two, three])
five = task_5.submit()
clean_up_task.submit(wait_for=[allow_failure(four), allow_failure(five)])
if __name__ == "__main__":
main()
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.