Are there any prohibitions/caveats associated with a non-task function in a flow, particularly with respect to how the framework might serialize the function and flow? As an example:
Copy code
def my_non_task_util_func(some_args):
return util_results
@task
def task_a(a_args):
return a_result
@task
def task_b(b_args):
return b_result
with Flow('test') as f:
util = my_non_task_util_func(some_args)
util_2 = my_non_task_util_func(some_other_args)
a = task_a(util, other_a_args)
b = task_b(util, util_2, a)
Should I be concerned about how Prefect treats this?
For further context, this util func is a factory for a collection of
related prefect.tasks.core.constants.Constant
s. Thus far, it has been easier to return these from a vanilla func rather than a task that returns a collection of tasks.
a
Amanda Wee
05/24/2021, 9:54 PM
If I understand correctly, the catch is that as a task, it is run when the flow runs, but as a function, it is run when the Python code for the flow is executed (e.g., at the time you build/register the flow).
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.