Hey folks, question for you all. Is it possible to encapsulate a series of tasks within one task in order to facilitate the writing of flows? Let’s say I have the following tasks A -> B -> C I want to create one task called D in such a way that if I create a flow E -> D, what I am really generating is E -> A -> B -> C Cheers!

Hey Maxime, You can definitely consolidate tasks into a single task, however you won’t have the granularity of control that you would normally have by assigning A, B, and C to their own tasks. Keep in mind that we do not have a higher-level API for reasoning about group tasks though, say if you wanted to add multiple tasks at once via

flow.add_task(big_task)

, you would only add one task to the flow despite three tasks existing in

big_task

.

Thanks @Kyle Moon-Wright, code wise, what would that look like?

Rather than encapsulating in a task, it seems you could encapsulate them in a plain old function:

@task
def a(x):
    return 2 * x

@task
def b(x):
    return x - 1

@task
def c(x):
    return x / 2

@task
def e():
    return 5
    
def abc(upstream):
    a_res = a(upstream)
    b_res = b(a_res)
    return c(b_res)

with Flow('test') as f:
    upstream = e()
    abc(upstream)

f.run()

This produces

E->A->B->C

, and it seems downstream tasks can be added to

C

via the returned value of

abc(upstream)

Indeed, if your goal is to see

A →B →C

in your flow, we’d recommend using a function to abstract the complexities of creating three tasks away (Prefect’s

ifelse

is a function that generates a few tasks for you, for example)

Thank you @emre and @Jeremiah! Really valuable input

@emre this is brilliant