Bradley Hurley
05/03/2022, 8:34 PMExecuteNotebook
Task.from prefect.tasks.jupyter.jupyter import ExecuteNotebook
execute_jupyter_notebook_task = ExecuteNotebook(output_format="json", log_output=True, path="<s3://bucket/dir/dir/dir/hello_world.ipynb>")
execute_jupyter_notebook_task()
Kevin Kho
Bradley Hurley
05/03/2022, 8:35 PMKevin Kho
Bradley Hurley
05/03/2022, 8:36 PMpath (string, optional): path to fetch the notebook from. Can be a cloud storage path. Can also be provided post-initialization by calling this task instance
import logging
import boto3
logger = logging.getLogger("papermill")
print("Hello World!")
<http://logger.info|logger.info>("Hello World!")
client = boto3.client("s3")
client.put_object(Bucket="Bucket", Key="path/output.txt"), Body="Success")
Kevin Kho
Bradley Hurley
05/03/2022, 8:39 PMKevin Kho
Bradley Hurley
05/03/2022, 8:43 PMKevin Kho
Bradley Hurley
05/03/2022, 8:46 PMKevin Kho
Bradley Hurley
05/04/2022, 5:05 PMpaepermill
logger via prefect.
• fixing the sparkcontext issues and enabling the logger via the env variable PREFECT__LOGGING__EXTRA_LOGGERS
corrected my issues