https://prefect.io logo
Title
n

Nikita Vostretsov

02/17/2020, 6:48 AM
Hi guys, I have next computational graph. It was easy to implement using functional API. Usually I want to run all five tasks. But, sometimes I want to run only upper branch (
t1
-
t3
) or lower branch (
t1
,
t4
and
t5
). Is it possible to do without rewriting code from
with Flow('flow') as flow:
   r1 = t1()
   r2 = t2(r1)
   r3 = t3(r2)
   r4 = t1(r1)
   r5 = t5(r4)
into
def upper_branch():
    r1 = t1()
    r2 = t2(r1)
    r3 = t3(r2)

def lower_branch():
    r1 = t1()
    r4 = t4(r1)
    r5 = t5(r2)

with Flow('flow') as flow:
    if do_upper or do_both:
        upper_branch()
    if do_lower or do_both:
        lower_branch()
t

trapped

02/17/2020, 2:57 PM
I believe you can use `prefect.tasks.control_flow.conditional.ifelse`: https://docs.prefect.io/core/task_library/control_flow.html#control-flow
j

Jeremiah

02/17/2020, 3:56 PM
Yup, this is exactly what the ifelse is for. You can use a Parameter to control which branch to execute.
n

Nikita Vostretsov

02/18/2020, 7:28 AM
From code readability point view this variant is not better then using separate functions and python's
if
. I was thinking about something like
with Flow('flow') as flow:
   r1 = t1()
   r2 = t2(r1)
   r3 = t3(r2)
   r4 = t1(r1)
   r5 = t5(r4)

targets = []
if do_upper or do_both:
   targets.append(r3)
if do_upper or do_both:
   targets.append(r5)
flow.run(targets=targets)
j

Jeremiah

02/18/2020, 4:29 PM
@Nikita Vostretsov Python’s
if
is not available inside a flow since it is executed at “build time”; Prefect’s
ifelse
is executed at “run time” and can therefore depend on information available when the flow runs. In your example, the flow actually runs both branches no matter what, as you have no runtime control flow implemented. The only thing changing is the input to your
targets
keyword argument, which is presumably a Parameter not shown here.
The pattern you’d like to implement is what
ifelse
is intended for; you want to make a decision at runtime about which branch to execute. If, on the other hand, you know which branch you want to run AND you don’t mind rebuilding your flow each time (which might be fine for your purposes!), then you could use a Python
if
to control which tasks you add to the flow. However, if you want a single flow with multiple branches, you’ll need a Prefect
ifelse
.