Jai P
04/02/2022, 5:06 PMflow
needs to be defined up front, but correct me if i'm wrong. I'll try to add a few more details in the thread, and thanks in advance for your help!def calculate_variable():
formula, inputs = get_formula_and_inputs()
input_values = retrieve_input_values()
return formula(*input_values)
this is a pretty trivial example, imagine that formula
actually has like n inputs and there are multiple transformation steps along the way (basically, formula
itself could be thought of as a flow
, but one that needs to be generated at runtime)@task
that encapsulates all that logic, or if i should be trying to do something else. I think in prefect 1.0, my only option is to have it as a @task
, but prefect 2.0 doesn't seem that way?Kevin Kho
case
taskJai P
04/02/2022, 6:00 PM@task
get_transformation():
return {
operation: "add",
input_1: "var_1",
input_2: "var_2"
}
@task
get_val_for_input(input):
return 1
@task
add_two_things(val_1, val_2):
return val_1 + val_2
@flow
do_transformation(transformation):
if transformation.operation == "add":
val_1 = get_val_for_input(transformation.input_1)
val_2 = get_val_for_input(transformation.input_2)
return add_two_things(val_1, val_2)
@flow
def do_the_thing():
transformation = get_transformation()
result = do_transformation(transformation)
return result
Kevin Kho
Anna Geller
imagine thatYou could solve it in 1.0 using mapping. But if you have the option of using Prefect 2.0, then that's definitely more appropriate.actually has like n inputsformula
Jai P
04/02/2022, 9:11 PMAnna Geller
@task
decorator with a flow decorator. This way, you can call it in any other flow. I believe this is a much better (and easier to reason about) pattern than calling tasks from other tasks. In the Radar view, you could drill down from a parent flow to a child flow (subflow), which makes it much easier to troubleshoot when something goes wrong, not even mentioning the modularity/composability of subflows making your workflow code cleaner and more reusable.Jai P
04/02/2022, 9:32 PM@task
with an @flow
, that was more of just a curiousity question π