https://prefect.io logo
Title
m

Marwan Sarieddine

07/10/2020, 3:36 PM
Question - in a simple local flow run, no parallelism involved, is there a way to make sure the flow run will always execute in some order ? (this helps for an ML reproducibility use-case I am thinking of … )
n

nicholas

07/10/2020, 3:38 PM
Hi @Marwan Sarieddine - yes, you can explicitly set the ordering of tasks using the
upstream_tasks
kwarg to a copy of the Task instance. You can read more about
upstream tasks
here: https://docs.prefect.io/api/latest/core/task.html#task-2
m

Marwan Sarieddine

07/10/2020, 3:38 PM
Hi @nicholas - thanks for the quick response - will take a look
ok so upstream_tasks by default is a set, but if I set it as a list then order is maintained (thanks!)… follow up question:
with Flow("test-flow"):
    f1 = task1a()
    task1b(f1)
    
    g1 = task2a()
    task2b(g1)
task1a
will always be executed before
task2a
in the simple local non-parallel run - correct ?
n

nicholas

07/10/2020, 4:21 PM
@Marwan Sarieddine - theoretically, yes, but you can ensure this by including
upstream_tasks=[f1]
in the constructor for
task2a
like this:
with Flow("test-flow") as flow:
    f1 = task1a()
    task1b(f1)
    g1 = task2a(upstream_tasks=[f1])
    task2b(g1)
m

Marwan Sarieddine

07/10/2020, 4:21 PM
oh ok - gotcha
n

nicholas

07/10/2020, 4:21 PM
You can see this in the flow DAG
m

Marwan Sarieddine

07/10/2020, 4:24 PM
yep - thank you !