https://prefect.io
Join Slack
Hi everyone, is there any suggestions to run task2 until task1 finished when they have no dependency...
# ask-community
z
Hi everyone, is there any suggestions to run task2 until task1 finished when they have no dependency?
Copy code
with Flow('feature-factory') as flow:
    a, b = pre_task()
    task1.map(a)
    task2.map(b)
It seems like an easy question. Currently my workaround solution for this is to add a meaningless dependency for them. Is there any better way to achieve this?
Copy code
with Flow('feature-factory') as flow:
    a, b = pre_task()
    state = task1.map(a)
    task2.map(b, state)
r
Think 
upstream_tasks
kwarg is what you’re looking for
z
Copy code
b.map(task2, upstream_tasks=[task1.map(a)])
I don’t know why task2 only run once (should be 9 runs), in my case, feature_generator=task1, dataset_generator=task2
And I find another workable solution
Copy code
a_task = task1.map(a)    a_task.set_downstream(task2.map(b))
k
Hey @Zac Chien, I think you want something like
Copy code
a = A.map(...)
b = B.map(..., upstream_tasks=[a])
If you want 
a
to fully complete before 
b
,
Copy code
a = A.map(...)
b = B.map(..., upstream_tasks=[unmapped(a)])
where 
unmapped
is imported from Prefect.
💪 1
z
thanks @Kevin Kho 🎉
👍 1
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by