thebuleon29

    thebuleon29

    1 year ago
    Hi guys, I am looking at the possibility to have loops in a flow. The existing LOOP feature that I found in the doc is interesting, but would it be possible to do the same with several tasks ? Like for example Task A produces a result, then Task B uses this result to produce another one, then Task C uses this second result, does its own thing with it and, if a condition is not met, calls A again. I know it sort of contradicts the meaning of a DAG, but it would be really useful for us... Is it an existing feature?
    Kevin Kho

    Kevin Kho

    1 year ago
    Hi @thebuleon29, it feels like it would be a lot simpler if you placed Task B and Task C inside the LOOP with Task A in this case. Does your use case prohibit this?
    thebuleon29

    thebuleon29

    1 year ago
    Sort of. We are reviewing workflow engines for a project, and one of the customer's requirements is to be able to do it. I managed to do it in Airflow using sub dags, so I am quite surprised to see Prefect not having an equivalent mechanism.
    Kevin Kho

    Kevin Kho

    1 year ago
    Could you explain the sort of a bit more? Just making sure you know that tasks can be used in other tasks by calling the
    task.run()
    method.
    Btw, I asked the team and one thing you can do here is have taskC start a new flow if the condition is not met. This would keep your tasks A, B, C as independent.
    thebuleon29

    thebuleon29

    1 year ago
    Hi, Thanks, starting a new flow was what I was looking for
    Kevin Kho

    Kevin Kho

    1 year ago
    This can be done through the Client where you can use the
    create_flow_run
    method if your condition is not met.