https://prefect.io logo
Title
i

Ian

01/19/2023, 7:55 PM
Hello all I'm pretty new to Prefect and have a question that I hope fits here. I tried searching for it here but didn't see anything related. We have a wrapper task around the task that Prefect provides for to run notebooks using papermill. It looks like when the papermill run fails, Prefect knows about and marks the task as failed. I assumed there is an exception being raised so I tried surrounding the call to run with try/except however was very surprised not to see any exception bubble up. Is there a different way that Prefect knows the papermill execution failed? Thank!!
k

Kasper Djernæs

01/20/2023, 2:15 PM
How do the logs look like from running the job? Is the problem that you do not see any error in the logs, but the job is still marked as failed?
i

Ian

01/20/2023, 2:26 PM
I am not seeing anything related to the error in the logs but I am able to see it in the "activity" view. And yes, the job/task is marked failed
k

Kasper Djernæs

01/20/2023, 2:27 PM
Can you paste the log or does it have sensitive information? Also, how do you execute the run?
i

Ian

01/20/2023, 2:29 PM
Unfortunately I don't think I can post the logs here. I am executing it by triggering from the UI
1
k

Kasper Djernæs

01/20/2023, 2:30 PM
But how does the code look like that executes the flow (or task) which you expect to contain and error?
i

Ian

01/20/2023, 2:39 PM
Ohh got you. I have a task class that is inheriting from ExecuteNotebook so it looks something like this: from prefect.tasks.jupyter.jupyter import ExecuteNotebook class MyExecuteNotebook(ExecuteNotebook): def run(self, **kwargs): try: super.run(...) except Exception as e: self.logger.exception("got error executing notebook: {e})
*sorry about the formatting. Typing on mobile and seeing an option to do code formatting on it.
k

Kasper Djernæs

01/20/2023, 2:56 PM
How do you define the logger?