👋 hello folks! I suspect the answer to this will be "use prefect 2.0" but wanted to ask: my company is investigating prefect for the use case of storing "DAG-y" transformations elsewhere (perhaps, behind some service interface), and dynamically executing them at runtime. I think prefect 1.0 won't solve this since the

flow

needs to be defined up front, but correct me if i'm wrong. I'll try to add a few more details in the thread, and thanks in advance for your help!

Yes that is right that Prefect 2.0 was specifically designed for the use case of adding tasks dynamically. The only tools you have for this in Prefect 1.0 are the conditionals like the

case

task

got it, thanks! so if i were to expand that example above a little, let's say that i get back a simple transformation that is represented as a json object would i write something like (in prefect 2.0):

@task
get_transformation():
    return {
        operation: "add",
        input_1: "var_1",
        input_2: "var_2"
    }

@task
get_val_for_input(input):
    return 1

@task
add_two_things(val_1, val_2):
    return val_1 + val_2

@flow
do_transformation(transformation):
    if transformation.operation == "add":
        val_1 = get_val_for_input(transformation.input_1)
        val_2 = get_val_for_input(transformation.input_2)
        return add_two_things(val_1, val_2)

@flow
def do_the_thing():
    transformation = get_transformation()
    result = do_transformation(transformation)
    return result

Yeah to can do that. It’s a lot trickier in Prefect 1 because subflows are not really first-class

imagine that

formula

actually has like n inputs

You could solve it in 1.0 using mapping. But if you have the option of using Prefect 2.0, then that's definitely more appropriate.

Hmmmm ok. I'll see if I can apply it to a more complex use case, but good to know that the subflow not being a first-class entity is problematic. I'm also correct in my understanding that I can't do tasks of tasks in prefect 1 right? Does that apply in prefect 2 also?

why not make it a subflow? in Prefect 2.0, you can replace the

@task

decorator with a flow decorator. This way, you can call it in any other flow. I believe this is a much better (and easier to reason about) pattern than calling tasks from other tasks. In the Radar view, you could drill down from a parent flow to a child flow (subflow), which makes it much easier to troubleshoot when something goes wrong, not even mentioning the modularity/composability of subflows making your workflow code cleaner and more reusable.

ahh thanks for clarification re: using an api call to trigger other flows! Definitely agree on replacing the

@task

with an

@flow

, that was more of just a curiousity question 🙂