https://prefect.io logo
s

Sébastien

12/09/2020, 11:46 AM
Hey all — I tried using
task2.set_upstream(task1)
to manually create a dependency, but that gives me:
Copy code
/usr/local/lib/python3.7/dist-packages/prefect/core/task.py:596: UserWarning: You are making a copy of a task that has dependencies on or to other tasks in the active flow context. The copy will not retain those dependencies.
What's the right way to say (in functional API) that task2 can only run after task1 was completed, without messing up the flow? The entire flow:
Copy code
schedule = CronSchedule("0 * * * *", start_date=datetime.now())

with Flow("Q", schedule=schedule) as flow:
    task2.set_upstream(task1)

    task1()
    some_result = task2()
    task3(some_result)
j

josh

12/09/2020, 1:56 PM
Hi @Sébastien you should define the tasks first before you try to set dependencies. Something like this:
Copy code
with Flow("Q", schedule=schedule) as flow:
    task_1 = task1()
    some_result = task2()    
    some_result.set_upstream(task_1)
    task3(some_result)
s

Sébastien

12/09/2020, 1:57 PM
@josh Shouldn't
task
have
set_upstream
rather than the result of a task?
j

josh

12/09/2020, 1:59 PM
Yeah that is the task, I was just copying your example. Could be better rewritten as:
Copy code
with Flow("Q", schedule=schedule) as flow:
    task_1 = task1()
    task_2 = task2()    
    task_2.set_upstream(task_1)
    task3(task_2)
Calling
task3(task2)
is a shorthand way of passing the result of task2 into task3 (and thus making is an upstream task). Calling
task_2.set_upstream(task_1)
is setting the upstream directly without passing any data.
s

Sébastien

12/09/2020, 2:01 PM
You call it a task, but use the return of calling the task. I'm assuming calling the decorator returns a Task-like thing rather than an actual return (since runtime is separate), and that's why you call both
task_2
and
task2
tasks?
For me, a
Task
is
@task
, which is
task2
but not
task_2
j

josh

12/09/2020, 2:03 PM
Yes you are correct, constructing a flow uses deferred computation so while it is technically the result of the task run at runtime, during flow creation it is seen as the task object
s

Sébastien

12/09/2020, 2:04 PM
Sweet, that clarifies things a bit. Thanks for clarifying my assumption. Now it makes sense why it considers my initial approach detached, since it actually branched by creating a copy of the task instead of attaching to one global flow.
At least that's my current best guess — I don't know much (yet) about the internals
@josh That fixed it, thanks!
👍 1
3 Views