It seems that Prefect doesn't support multiple ret...
# prefect-community
g
It seems that Prefect doesn't support multiple return, or some Python syntax? ----------------------------------- @task def hello(x): a=1 b=2 return a, b with Flow('test flow') as flow: a, b = hello(1) #<---- 'NoneType' object is not callable hello(a) hello(b) ______________________________
j
Hi @Gary Liao, Prefect is building a deferred computational pipeline. That means that at the time you call the function
hello()
, Prefect isn’t running the function and doesn’t know that it’s going to return two items. Therefore, you can’t use Python unpacking assignment to unpack a and b inplace.
Instead, you can do this to achieve the same thing:
Copy code
hello_result = hello(1)
a = hello_result[0]
b = hello_result[1]
...
Prefect can transform those indexes into deferred computations
g
Thank you for replying so fast! I know the approach you provide works, but how to avoid the visualize function to grow so many "atomic" flows?
e
You ‘could’ build your own
GetItem
task:
Copy code
class MyGetItem(Task):
    def __init__(self, key, *args, **kwargs):
        self.key = key
        super().__init__(*args, **kwargs)
    def run(self, task_result):
        return task_result[self.key]
then call as such:
Copy code
hello_result = hello(1)
a = MyGetItem(0)(hello_result)
b = MyGetItem(1)(hello_result)
Obviously it is uglier, and also requires you to declare the indexes outside of the flow. You can’t use
MyGetItem(a)
, where
a
is an upstream task result.
j
@emre that’s actually what Prefect is doing behind the scenes 🙂
Prefect will also auto-generate classes to represent all operators — for example if you do
Task() + Task()
it’ll generate an
Add()
task for you, or if you pass
[Task(), Task()]
as the input to another task, it’ll automatically generate a
List()
task. I’m not sure we’ve really publicized that
b
this might be a very bad idea, but is it maybe feasible to support unpacking by overloading
__iter__
for tasks? the GetItem behavior is great but the
*
syntax is just so much prettier 🙂
😃 2
j
This is the sort of message that results in me staying up all night 😉
We tried that actually
For a reason I don’t recall specifically, it creates a recursion issue due to how we iterate over tasks in a different part of the codebase. Maybe operator overloads? I’ll have to go check! But we’re also older and wiser and maybe we can figure it out now 🙂
🧙‍♂️ 1
e
Copy code
that's actually what Prefect is doing behind the scenes
Oh I know, went a little source code mining for this myself 😅 Anyways, all aboard the unpacking hype train 🚂
j
🤓