a

    An Hoang

    7 months ago
    I have a
    flow_A
    with
    task1, task2... task10
    . Now I want to construct
    flow_B
    with
    task1, task2.... task 20
    and
    task11_result = task11(task10_result)
    . How do I add the tasks of
    flow_A
    as upstream tasks for
    flow_B
    . I know I can do
    create_flow_run
    , but I do not want monitoring for
    task1...task10
    in
    flow_B
    Kind of like:
    with Flow("flow_A") as flow:
        flow_A_param1 = Parameter("flow_A_param1")
        ...
        flow_A_param_n = Parameter(...)
        ...
        task1 = ....
        ...
        task10_result = task10(...)
    
    with Flow("flow_B") as flow:
        flow_A_param1 = Parameter(...)
        ...
        flow_A_param_n = Parameter(...)
        
        flow_A_result = flow_A.run(parameters = {"flow_A_param1": flow_A_param1,...., "flow_A_param_n": flow_A_param_n}) #dont want monitoring for these tasks
        task10_result = flow_A_result.result[flow_A.get_task(task10)]
        task11_result = task11(task10_result)
        ...
        task20_result = task20(...)
    where flow_B's parameters are just flow_A's parameters. But in the example above I always have to repeat the parameters of
    flow_A
    in
    flow_B
    and keep them both up to date with each other. Is there a better way to do this?
    Kevin Kho

    Kevin Kho

    7 months ago
    Oh man this is hard. This is all Prefect Core right? So there is no GraphQL API to query?
    a

    An Hoang

    7 months ago
    Yes
    flow_A
    would be ran using prefect core and
    flow_B
    would be ran using Prefect cloud for monitoring
    I guess the first thing to tackle is how to make flow share the same parameters and then add tasks that build upon those parameters
    Kevin Kho

    Kevin Kho

    7 months ago
    You would need a function to create the Flows and return them at the very least to make it dynamic based on inputs