Andreas Nord

    Andreas Nord

    3 months ago
    What is the best way of restricting some tasks (with "tag2") to not run until all tasks (with "tag1") has succeeded? Does not have to be tag-based (Prefect 1)
    Kevin Kho

    Kevin Kho

    3 months ago
    There really isn’t any mechanism other than the
    set_upstream
    and
    upstream_tasks
    . For this. You can maybe just keep a list as you build your flow and then set
    task_with_tag_2(..., upstream_tasks=[list_of_tag_one_items])
    Andreas Nord

    Andreas Nord

    3 months ago
    Right, thought there was an easier way. With Prefect 2 I can nest flows and tasks and solve it that way?
    Kevin Kho

    Kevin Kho

    3 months ago
    Maybe. How would you represent it in native Python?
    Andreas Nord

    Andreas Nord

    3 months ago
    Something like (not sure about the syntax):
    def task1():
        # all tasks that had tag1
        task1a()
        task1b()
        ... 
    # In flow:
    task1()
    task2(upstream_tasks=[task1])
    Kevin Kho

    Kevin Kho

    3 months ago
    It’s not quite as elegant as that because you can’t make the native Python function there an upstream tasks
    Andreas Nord

    Andreas Nord

    3 months ago
    But assuming it is a task it should work?
    Kevin Kho

    Kevin Kho

    3 months ago
    But you can’t run a task inside a task either 😅 so the earlier code will not work. It’s going to be something more like:
    def task1():
        # all tasks that had tag1
        a = task1a()
        b = task1b()
        ...
        return [a,b,c]
     
    # In flow:
    up = task1()
    task2(upstream_tasks=up)
    Which will surely work in Prefect 2. But it might already work in Prefect 1