Hi Prefect Experts Im looking for a little guidance on using

Question

Hi Prefect Experts Im looking for a little guidance on using the `ExecuteNotebook` Task

Bradley Hurley · Answer

I have a really basic test flow that is something like ```from prefect tasks jupyter jupyter import ExecuteNotebook execute jupyter notebook task = ExecuteNotebook output format= json log output=True path= <s3 bucket dir dir dir hello world ipynb> execute jupyter notebook task ```

Kevin Kho · Answer

What is the question Are you trying to use it outside a Flow

Bradley Hurley · Answer

Im using it inside a flow but I just wanted to really strip it down

Bradley Hurley · Answer

I can t seem to get any logs and it doesn t actually seem to be running

Kevin Kho · Answer

Ah I see Are you sure you can pass an s3 path

Bradley Hurley · Answer

The docs say ```path string optional path to fetch the notebook from Can be a cloud storage path Can also be provided post initialization by calling this task instance```

Bradley Hurley · Answer

My notebook only contains this ```import logging import boto3 logger = logging getLogger papermill print Hello World <http logger info|logger info> Hello World client = boto3 client s3 client put object Bucket= Bucket Key= path output txt Body= Success ```

Kevin Kho · Answer

I don t have any better suggestion except to try the papermill code under the hood and see if it works <https github com PrefectHQ prefect blob 40e2e2bd1c1de5651bc7fb484c823e43cfc69877 src prefect tasks jupyter jupyter py L74 L85|here>

Bradley Hurley · Answer

Thanks I searched the slack here and the internet and wasn t able to find much

Kevin Kho · Answer

Not a ton of people use this but I wrote the tests for this before and I think it works on local notebooks for sure

Bradley Hurley · Answer

I did see your name on <https github com PrefectHQ prefect issues 4265> and the linked merged PR

Kevin Kho · Answer

Yes that was my first or second week here Let s not put that much faith in me either sweat smile Would not be surprised if a papermill upgrade caused issues but I never tested on S3

Bradley Hurley · Answer

I will report back with my findings It might take a day

Kevin Kho · Answer

Thank you

Bradley Hurley · Answer

The followup on this was multi pronged We are using and the first thing that happens when a cell is executed is that sparkmagic attempts to create a sparkcontext The task was reporting success even though when the logs were retrieved from the kubernetes job pod it showed the actual error message I overlooked the instructions in the and didn t enable the `paepermill` logger via prefect fixing the sparkcontext issues and enabling the logger via the env variable `PREFECT LOGGING EXTRA LOGGERS` corrected my issues