William Jamir
06/14/2022, 9:01 AMfrom prefect import Flow, task
@task
def task_1():
task_2.run()
task_3.map(range(2))
@task
def task_2():
print('2')
@task
def task_3(x):
print(x)
with Flow(name='a') as flow:
task_1()
flow.run()
When running this code I’m receiving
Could not infer an active Flow context while creating edge to <Task: task_3>.
This often means you called a task outside a `with Flow(...)` block. If you're trying to run this task outside of a Flow context, you need to call `task_3.run(...)`
Anna Geller
06/14/2022, 11:16 AMtask_3.map(range(2))
also, you really shouldn't have to do:
task_2.run()
instead, call task_2() in your FlowWilliam Jamir
06/14/2022, 12:59 PMyou need to call mapping within your Flow, not within a task, so this line needs to move to FlowAh okay, so sorry if I did not make my intention clear. I explicitly want to call a map from inside a task, in order (if possible) to make my task composable. So, is impossible/limitation to call a
map
inside a task? (like we do with task_2.run()).
instead, call task_2() in your FlowYeap, indeed, sorry if my intention was not clear. I only called
task_2
like this in order to show that I was trying to compose my tasks to be reusable.
Taking advantage of this message, can I execute tasks like this (with run inside another task), or its not recommended?
I mean, is there any drawback that I should be aware on calling tasks like this?Anna Geller
06/14/2022, 1:03 PMis impossible/limitation to call aIt's not a limitation, it's an intentional design decision from Prefect engineers to trigger mapped tasks from a flow rather than from a task - what are you trying to do here? if you want to chain them so that mapped runs after it, you would useinside a task? (like we do with task_2.run())map
upstream_tasks
keyword
if the problem is to make some code reusable, you can build it as custom functions that you can package into a Python package/module and then call from within your taskscan I execute tasks like this (with run inside another task), or its not recommended?Great question - it's indeed not recommended to run tasks from other tasks. The drawback is lack of visibility - you don't see such a task in any of your workflow schematics, makes your code less composable since it's not clear what calls what, and makes it harder to maintain since there is a mix of responsibility - the task is no longer responsible for one thing. Prefect encourages small tasks since this makes things easier to build, troubleshoot and maintain