https://prefect.io logo
Title
k

Kyle McEntush

08/22/2020, 12:07 AM
Is there a way to run functions that are decorated with @task outside of prefect? I'm trying to rapidly develop individual task stages but I'd prefer to not have two functions defined. Currently, calling a function decorated with @task in a jupyter notebook throws a context error.
c

Chris White

08/22/2020, 12:08 AM
Hi @Kyle McEntush - absolutely. E.g.,:
@task
def my_function(x, y):
    return x + y

my_function.run(1, 2) # runs with no Prefect intervention
👍 1
1
k

Kyle McEntush

08/22/2020, 12:08 AM
Got it, thanks!
👍 1
c

Chris White

08/22/2020, 12:09 AM
@Marvin archive “How to run a Prefect Task without Prefect”
k

Kyle McEntush

08/22/2020, 1:16 AM
Interesting. Calling .run() on a task that lives within a class doesn't automatically pass the "self" argument.
Is that intentional?
c

Chris White

08/22/2020, 1:16 AM
Can you share a traceback? Did you initialize the class before calling
.run()
?
k

Kyle McEntush

08/22/2020, 1:18 AM
from prefect import task

class Test:
    def __init__(self):
        pass

    @task
    def test_func(self, val):
        return val
    
test = Test()
test.test_func.run(1)
Throws
TypeError: test_func() missing 1 required positional argument: 'val'
c

Chris White

08/22/2020, 1:19 AM
I see - this is not a supported pattern; all arguments to a task need to be capable of being converted to tasks themselves. To achieve the same thing you should do:
from prefect import Task

class Test(Task):
    def run(self, val):
        return val

test = Test()
test.run(1)
k

Kyle McEntush

08/22/2020, 1:20 AM
in that setup, can you have multiple functions that can be run?
c

Chris White

08/22/2020, 1:20 AM
What do you mean?
k

Kyle McEntush

08/22/2020, 1:20 AM
from prefect import Task

class Test(Task):
    def run(self, val):
        return val

    def run2(self, val):
        return val

test = Test()
test.run(1)
test.run2.run(1)
oops, second
I presume that won't work for the same reasons as before
c

Chris White

08/22/2020, 1:22 AM
What are you trying to achieve with this setup?
k

Kyle McEntush

08/22/2020, 1:22 AM
The end goal is to have multiple tasks defined within a class
but it seems that goes against the design of prefect
c

Chris White

08/22/2020, 1:22 AM
but to what end? Why do the tasks need to all be methods of a single class?
k

Kyle McEntush

08/22/2020, 1:23 AM
many of the tasks share common arguments, so I figured a class would be ideal
But within the current structure I could always pass a dict through to mimic the self arg
c

Chris White

08/22/2020, 1:24 AM
gotcha; because Prefect supports distributed environments, the only way to share state between tasks is via arguments <-> return values, so yea a dict of data makes sense
k

Kyle McEntush

08/22/2020, 1:24 AM
Okay, thanks for the tip
c

Chris White

08/22/2020, 1:24 AM
👍 anytime
k

Kyle McEntush

08/22/2020, 1:24 AM
appreciate your responsiveness at 6:30pm on a Friday 🙂
c

Chris White

08/22/2020, 1:25 AM
haha 🍻