https://prefect.io logo
Title
s

Seth Goodman

10/21/2022, 3:50 PM
Hi All - hopefully an easy question: What is the proper way to redefine an existing task? E.g., In interactive mode I create my_task, then edit my_task, and I get an error of "OSError: could not get source code". Using 2.6.4
1
j

Jean Luciano

10/21/2022, 4:12 PM
Did you build the deployment after making changes to your task?
s

Seth Goodman

10/21/2022, 4:17 PM
I rebuilt the flow the task was running in before calling the flow (editing/redefining the flow instead of the task presents the same error), but I haven't created an actual deployment of the flow - I have just been testing the functionality in an interactive python session
@Jean Luciano barebones example to replicate:
from prefect import task, flow

@task
def my_task():
    print("hello")

@flow
def my_flow():
    my_task.submit()

my_flow()
the error is when i then try to do:
@task
def my_task():
    print("goodbye")

# error here after trying to redefine task

@flow
def my_flow():
    my_task.submit()

my_flow()
j

Jean Luciano

10/21/2022, 4:48 PM
Do you get the same error by calling the task directly i.e
my_task()
?
s

Seth Goodman

10/21/2022, 4:50 PM
the error is when just defining the task, not actually trying to call it
so even if i just put the exact same task function into my terminal twice i would get the error on the second one
@task
def my_task():
    print("hello")

@task
def my_task():
    print("hello")

# error trigger by second instance
j

Jean Luciano

10/21/2022, 5:10 PM
Is this using ipython?
s

Seth Goodman

10/21/2022, 5:13 PM
just standard shell interpreter
r

Ryan Peden

10/21/2022, 9:08 PM
This happens in interactive mode because when you create a new flow or task with the same name as an existing flow or task, Prefect tries to load the file where the redefinition happened so it can give you a friendly warning message. But, of course, that doesn't work if your code isn't in a file 🙂 Normally, you wouldn't need to redefine an existing task because there's usually a better way to accomplish your goals. But if you just want to do a few quick experiments in an interactive session, I can see why redefining and re-running tasks would be handy. A quick way to work around this would be giving the tasks different names so Prefect doesn't see the redefinition as a duplicate. In that case, your sample would look like:
@task(name="1")
def my_task():
    print("hello")

@task(name="2")
def my_task():
    print("hello")

# Prefect remains happy
If you want to get really creative, you could run something like this at the beginning of your interactive Python session:
import uuid
from prefect import task as _task
task = lambda fn, **kwargs: _task(fn, name=str(uuid.uuid4()), **kwargs)

# then, these should work
@task
def my_task():
    print("hello")

@task
def my_task():
    print("hello")

# Prefect still happy
Disclaimer: you should definitely not do anything like that in production code. But if you just want a quick way to redefine a task a couple of dozen times when you're experimenting in a Python REPL, it should be okay.
👍 1