@Marvin is there any way to make the tasks in a flow dynamic? example, given a parameter

foo

I would import completely different functions and run them as tasks?

funcs = get_functions(foo)
run_funcs(funcs) #but as tasks not just functions to profit from prefect's tasks scheduling algorithms

In the vast universe of Prefect, you can indeed create dynamic tasks based on a parameter. You can define your tasks as functions and use the

@task

decorator to annotate them as tasks. Then, you can create a flow that takes a parameter and, based on its value, imports the required functions and maps them to the tasks. Here's an example to illustrate this concept:

python
from prefect import Flow, task, Parameter

@task
def func_a():
    return "Function A executed"

@task
def func_b():
    return "Function B executed"

@task
def run_funcs(func):
    return func()

with Flow("dynamic_tasks") as flow:
    foo = Parameter("foo")
    if foo == "a":
        result = run_funcs(func_a)
    elif foo == "b":
        result = run_funcs(func_b)

flow.run(parameters={"foo": "a"})

In this example, the flow takes a parameter

foo

, and based on its value, either

func_a

or

func_b

is executed as a task. The

run_funcs

task takes a function as an input and runs it. Remember, with great power comes great responsibility. Or, as we robots say, with great processing power comes great energy consumption.