Hello! In our work we use Jupyter notebooks a lot, especially for generating reports at the end of a task (e.g. analyzing classifier performance across categories with confusion matrices and whatnot). So I wrote a Prefect

JupyterTask

for internal use, and was wondering if anyone would be interested in such a thing before I make the extra effort of publishing it in a package

Hi Avi, we’ve just begun to use Prefect in our data engineering and analytics workflows, but would definitely interested in such functionality!

This sounds awesome! I’d suggest creating a PR and adding it to the Task Library as opposed to your own package though! That way other Prefect users can locate it easily!

Hey @Avi A there’s definitely some interest! The good news, like Alex said, is that you don’t have to publish your own package! This is perfect for our Task Library. Here’s our contributing guide and a link to the task library: https://docs.prefect.io/core/development/contributing.html https://docs.prefect.io/core/task_library/ Thank you for helping us make Prefect even better!

I’ll definitely take a JupyterTask, please and many thanks! 😄

Thanks for the feedback guys, I hope to submit a PR sometime in the next few weeks. I hope @Marvin will be there to approve

I would like to say that it is a very great pleasure, honour and privilege for me to enter @Avi A in this contest, but I can’t because my lying circuits are all out of commission.

Ok took longer than expected but here it is https://github.com/PrefectHQ/prefect/pull/3599 Task currently allows you to get the jupyter notebook output as either HTML or the original JSON, which you can then save using the

Result

handler or pass the output to one of the cloud upload tasks (e.g.

GCSUpload

). LMK if you think it should be given any further enhancement

Hey @Avi A that's great, thanks for the update! Looking forward to trying it

Thanks for sharing, @Avi A! Looks like this fits our use case perfectly 🙂