John Faucett
10/21/2019, 7:09 PMChris White
10/21/2019, 7:17 PMJohn Faucett
10/21/2019, 7:19 PMChris White
10/21/2019, 7:20 PMwith Flow("parameter-example") as flow:
p, q = Parameter("p"), Parameter("q")
result = my_task(input_kwarg=p, another_kwarg=q)
John Faucett
10/21/2019, 7:21 PMmy_task.set_upstream(my_param, flow=flow, key='my_param_name')
Chris White
10/21/2019, 7:22 PMmy_task
accepts a my_param_name
keyword argument in its run
signatureJohn Faucett
10/21/2019, 7:22 PMmy_task
and it will just work?Chris White
10/21/2019, 7:25 PMJohn Faucett
10/21/2019, 7:26 PMChris White
10/21/2019, 7:28 PMtask_b.set_upstream(task_a, flow=flow)
in this case, no data is exchanged but there is now an enforced dependency relationship. You can also see these relationships with flow.visualize()
for easier introspectionJohn Faucett
10/21/2019, 7:29 PMtask_b.set_upstream(task_a, flow=flow)
task_b.set_upstream(task_c, flow=flow)
will it wait for both upstreams to finish?Chris White
10/21/2019, 7:29 PMJohn Faucett
10/21/2019, 7:30 PMChris White
10/21/2019, 7:31 PMJohn Faucett
10/21/2019, 7:34 PMChris White
10/21/2019, 7:34 PMJohn Faucett
10/21/2019, 7:39 PM