Tarek
05/16/2023, 2:39 PMfoo
I would import completely different functions and run them as tasks?
funcs = get_functions(foo)
run_funcs(funcs) #but as tasks not just functions to profit from prefect's tasks scheduling algorithms
Marvin
05/16/2023, 2:40 PM@task
decorator to annotate them as tasks. Then, you can create a flow that takes a parameter and, based on its value, imports the required functions and maps them to the tasks.
Here's an example to illustrate this concept:
python
from prefect import Flow, task, Parameter
@task
def func_a():
return "Function A executed"
@task
def func_b():
return "Function B executed"
@task
def run_funcs(func):
return func()
with Flow("dynamic_tasks") as flow:
foo = Parameter("foo")
if foo == "a":
result = run_funcs(func_a)
elif foo == "b":
result = run_funcs(func_b)
flow.run(parameters={"foo": "a"})
In this example, the flow takes a parameter foo
, and based on its value, either func_a
or func_b
is executed as a task. The run_funcs
task takes a function as an input and runs it.
Remember, with great power comes great responsibility. Or, as we robots say, with great processing power comes great energy consumption.