https://prefect.io
Join Slack
Hi, How can I execute a map inside a task? Something like this (code in thread):
# ask-community
w
Hi, How can I execute a map inside a task? Something like this (code in thread):
✅ 1
I’m using Prefect 1
Copy code
from prefect import Flow, task


@task
def task_1():
    task_2.run()
    task_3.map(range(2))


@task
def task_2():
    print('2')


@task
def task_3(x):
    print(x)


with Flow(name='a') as flow:
    task_1()

flow.run()
When running this code I’m receiving
Copy code
Could not infer an active Flow context while creating edge to <Task: task_3>.
This often means you called a task outside a `with Flow(...)` block. If you're trying to run this task outside of a Flow context, you need to call `task_3.run(...)`
a
you need to call mapping within your Flow, not within a task, so this line needs to move to Flow
Copy code
task_3.map(range(2))
also, you really shouldn't have to do:
Copy code
task_2.run()
instead, call task_2() in your Flow
also @William Jamir can you move the code into the thread?
w
@Anna Geller Sure, code moved to thread.
you need to call mapping within your Flow, not within a task, so this line needs to move to Flow
Ah okay, so sorry if I did not make my intention clear. I explicitly want to call a map from inside a task, in order (if possible) to make my task composable. So, is impossible/limitation to call a 
map
inside a task? (like we do with task_2.run()).
instead, call task_2() in your Flow
Yeap, indeed, sorry if my intention was not clear. I only called 
task_2
like this in order to show that I was trying to compose my tasks to be reusable. Taking advantage of this message, can I execute tasks like this (with run inside another task), or its not recommended? I mean, is there any drawback that I should be aware on calling tasks like this?
a
is impossible/limitation to call a 
map
inside a task? (like we do with task_2.run())
It's not a limitation, it's an intentional design decision from Prefect engineers to trigger mapped tasks from a flow rather than from a task - what are you trying to do here? if you want to chain them so that mapped runs after it, you would use 
upstream_tasks
keyword if the problem is to make some code reusable, you can build it as custom functions that you can package into a Python package/module and then call from within your tasks
can I execute tasks like this (with run inside another task), or its not recommended?
Great question - it's indeed not recommended to run tasks from other tasks. The drawback is lack of visibility - you don't see such a task in any of your workflow schematics, makes your code less composable since it's not clear what calls what, and makes it harder to maintain since there is a mix of responsibility - the task is no longer responsible for one thing. Prefect encourages small tasks since this makes things easier to build, troubleshoot and maintain
9 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by