Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hey folks, just trying out prefect for the first time after using some other workflow systems before, loving the UX! Quick question, I’m using the `ExecuteNotebook` task and it’s spilling the cell content to stdout during normal execution, is there an option to supress? I’m using a target file to cache the output and that’s working fine

Hey <@U02C1HAH4AK>, glad you are enjoying Prefect. Do you hvae `log_output=True` by chance?

```        notebook_task = ExecuteNotebook(
            str(nb_path),
            kernel_name="dev",
            parameters={**opts},
            result=LocalResult(dir=cachedir, serializer=PlainTextSerializer()),
            target="output.ipynb",
        )```
this is how I’m calling it, PlainTextSerializer is just a simple serializer i wrote to dump strings as-is

Canb you try adding `log_output=False` to `ExecuteNotebook` and see what happens?

also if there’s a way to pipe this ooutput directly to my target file that would be even better

Maybe you can try the different output_formats?
```        - output_format (str, optional): Notebook output format, should be a valid
            nbconvert Exporter name. 'json' is treated as 'notebook'.
            Valid exporter names: asciidoc, custom, html, latex, markdown,
            notebook, pdf, python, rst, script, slides, webpdf. (default: notebook)```

Not sure which one applies to you, but I guess if there is a way to do it with papermill that we don’t expose, you could open an issue for it.

tried a bunch of combinations but nothing quite does what i want

I might just template markdown and stop using papermill TBH

Just curious, do you have experience with papermill before this? Wondering if the task can be improved.

only a little, this doesn’t happen when i run papermill manually

but it is pretty big issue IMO to have the logs polluted by binary output data in stdout

Gotcha, PR’s are definitely welcome if you’re interested :slightly_smiling_face:

```        nb: nbformat.NotebookNode = pm.execute_notebook(
            path,
            "-",
            parameters=parameters,
            kernel_name=self.kernel_name,
            log_output=self.log_output,
        )```
I think this is the cause here, in `prefect.tasks.jupyter.jupyter.ExecuteNotebook.run`

the `"-"` argument is being used as the output path which when i run on the CLI pipes to stdout it seems

i can take a gander this weekend and try to open a Pr