An Hoang
02/02/2022, 5:38 PMflow_A
with task1, task2... task10
. Now I want to construct flow_B
with task1, task2.... task 20
and task11_result = task11(task10_result)
. How do I add the tasks of flow_A
as upstream tasks for flow_B
. I know I can do create_flow_run
, but I do not want monitoring for task1...task10
in flow_B
Kind of like:
with Flow("flow_A") as flow:
flow_A_param1 = Parameter("flow_A_param1")
...
flow_A_param_n = Parameter(...)
...
task1 = ....
...
task10_result = task10(...)
with Flow("flow_B") as flow:
flow_A_param1 = Parameter(...)
...
flow_A_param_n = Parameter(...)
flow_A_result = flow_A.run(parameters = {"flow_A_param1": flow_A_param1,...., "flow_A_param_n": flow_A_param_n}) #dont want monitoring for these tasks
task10_result = flow_A_result.result[flow_A.get_task(task10)]
task11_result = task11(task10_result)
...
task20_result = task20(...)
where `flow_B`'s parameters are just `flow_A`'s parameters. But in the example above I always have to repeat the parameters of flow_A
in flow_B
and keep them both up to date with each other. Is there a better way to do this?Kevin Kho
An Hoang
02/02/2022, 6:10 PMflow_A
would be ran using prefect core and flow_B
would be ran using Prefect cloud for monitoringKevin Kho