I remember reading a Prefect doc about how to set additional reference tasks (other than default fin...

Hugo Kitano

07/28/2021, 6:15 PM

I remember reading a Prefect doc about how to set additional reference tasks (other than default final one) but can’t find it now. How would I add a reference task to a flow?

Sam Cook

07/28/2021, 6:19 PM

You can either pass the tasks in through the Flow constructor OR use the set_reference_tasks method on the flow https://docs.prefect.io/api/latest/core/flow.html#flow-2

upvote 2

Kevin Kho

07/28/2021, 6:24 PM

Sam is spot on. It would be

flow.set_reference_tasks([c])

where c is a task inside your Flow.

👍 1

Hugo Kitano

07/28/2021, 8:40 PM

@Kevin Kho One thing I’ve found is c has to be an output of a task, so if I have a task that I want to set as a reference

Copy code

output = reference_task(input)

I have to set the reference task as

Copy code

flow.set_reference_tasks(tasks = [output])

which to me, is a bit clunky. Is there a way to specify the task and not the output as an argument to the

set_reference_task

method?

Kevin Kho

07/28/2021, 8:45 PM

Unfortunately, no, only the output can be passed.

Hugo Kitano

07/28/2021, 8:46 PM

what about with task slugs?

Kevin Kho

07/28/2021, 8:50 PM

Let me check

Kevin Kho

07/28/2021, 9:09 PM

It might be ugly. I think this works:

Copy code

with Flow("xxx") as flow:
    a = abc(1)
    b = abc(2, upstream_tasks=[a])
    bcd(2)

flow.set_reference_tasks([flow.get_tasks(name="bcd")[0]])

Hugo Kitano

07/28/2021, 9:10 PM

ok, thanks. I see. may just stick with the first option then haha

😅 1

6 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.