Hello everyone! Just a noobie question here, how do I create edges (one task can only execute if the previous is successful) between tasks that don't pass "results" between them? For example, let suppose that I have a program that creates some folders, login to an API, and collects some results being:
Create Folder 1 > Create Folder 2 > Login to API > Collect Results
The three initial tasks don't return results, but I want to run in "chained" execution.
The docs @Thomas Weatherston will help, but here is the syntax
Copy code
@task()
def abc(a):
return a
with Flow("ecs_testing") as flow:
x = abc(1)
y = abc(1, upstream_tasks=[x])
with Flow("ecs_tes") as flow2:
x = abc(1)
y = abc(1)
y.set_upstream(x)
π 1
c
codingtacos
08/12/2021, 3:15 PM
Is
flow.add_edge()
no longer used?
k
Kevin Kho
08/12/2021, 3:17 PM
You could do that if it suits you. Do you use that along with the functional API or do you use the imperative API?
π 1
c
codingtacos
08/12/2021, 3:23 PM
Using both upstream/edge and func/imper api - I need to land on a style for sure π
i
Italo Barros
08/12/2021, 3:48 PM
Hey @Kevin Kho, the task must return some variable to be properly upstreamed?
k
Kevin Kho
08/12/2021, 3:50 PM
If you only reference it once like
abc()
, I think you can do
upstream_tasks=[abc]
, but of course when you do it multiple times, you should probably use the variable assignment. And no you donβt need to return anything. That variable is just a reference for the Task, and the result can be None
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.