Hello everyone, we have been testing Prefect since December, and so far we really like it and are excited to test it on live projects. I am by now quite familiar with the docs and with orchestration, and I would like to ask for some help on one thing which is not entirely clear yet to me. I want to give an example: Let’s say we have a flow that does SQL stuff on Postgres, for example

1) create table a

,

2) create table b

and

3) join data from tables a and b

. We want to use the dask executor, because there are numerous more tasks. What is the recommended way to have

task 1

and

task 2

run simultaneously, but have

task 3

wait for tasks 1 and 2? The dask executor would create a DAG where it would run tasks 1, 2 and 3 together, but that will fail, since task 3 is depending on tasks 1 and 2. The Python code however, has no dependency. I have few ideas, and I am excited to see your opinion on that question. I hope that I could express myself clearly. Best regards from Berlin, David.

Hi @David Kuda you can set task upstreams and downstreams directly through the API with something like this:

In [1]: with Flow("my_flow") as flow:
   ...:     t1 = my_task1()
   ...:     t2 = my_task2()
   ...:     t3 = my_task3()
   ...:
   ...:     t3.set_upstream(t1)
   ...:     t3.set_upstream(t2)

Hey @josh — Thank you so much for your fast and helpful reply, that’s great! Do you have a link to the docs that gives further details on that?

Hi @David Kuda — there’s an example at https://docs.prefect.io/core/getting_started/first-steps.html#imperative-api

Thanks, Michael 🙂 Additionally, I have just found: https://github.com/PrefectHQ/prefect/blob/46afb3a394da1d9a3ff4481b66666f487a312041/src/prefect/core/task.py#L767 and: https://docs.prefect.io/api/latest/core/task.html#task-2

Yep the documentation and clean look to the code is what has attracted me to Prefect.