Hey guys. New to prefect, but already love it. I was wondering if there is a way to debug/view the intermediate result of a task during the flow.run() call? Some task in my pipelines messes up my final output and I would like to be able to quickly tab over the intermediate results of the tasks to see which is causing this behavior. Thank you!

Hey @Lucas Fobian! You can log it like :

@task
def abc():
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>(the_value)
    return 1

Hey @Kevin Kho. Thank you for your quick response! My tasks mostly manipulate a DataFrame - I thought of something like being able to view the returned python object from a task directly while the flow is running (in debug mode). Printing/logging a DF is a little bit tedious since not all rows/columns are displayed.

If you return the DataFrame, you can run a task by calling

task.run()

. If you’re in a notebook/interactive mode, maybe you can just run the task to diagnose?

Yes, thats what I'm using in my unittests (but there I'm using a subset of the full DF). I just thought there would be a super straight forward answer to this that I might be missing. But if there isn't than it's fine - I actually used breakpoints at the end of each task to tab through the output of each. That will work aswell I guess. Thank you again for your quick response!