Thread
#prefect-community
    Zac Chien

    Zac Chien

    1 year ago
    Hi everyone, is there any suggestions to run task2 until task1 finished when they have no dependency?
    with Flow('feature-factory') as flow:
        a, b = pre_task()
        task1.map(a)
        task2.map(b)
    It seems like an easy question. Currently my workaround solution for this is to add a meaningless dependency for them. Is there any better way to achieve this?
    with Flow('feature-factory') as flow:
        a, b = pre_task()
        state = task1.map(a)
        task2.map(b, state)
    r

    Robert Hales

    1 year ago
    Think
    upstream_tasks
    kwarg is what you’re looking for
    Zac Chien

    Zac Chien

    1 year ago
    b.map(task2, upstream_tasks=[task1.map(a)])
    I don’t know why task2 only run once (should be 9 runs), in my case, feature_generator=task1, dataset_generator=task2
    And I find another workable solution
    a_task = task1.map(a)    a_task.set_downstream(task2.map(b))
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Zac Chien, I think you want something like
    a = A.map(...)
    b = B.map(..., upstream_tasks=[a])
    If you want
    a
    to fully complete before
    b
    ,
    a = A.map(...)
    b = B.map(..., upstream_tasks=[unmapped(a)])
    where
    unmapped
    is imported from Prefect.
    Zac Chien

    Zac Chien

    1 year ago
    thanks @Kevin Kho 🎉