André Lynum
07/03/2026, 8:35 AMMarvin
07/03/2026, 8:35 AMMarvin
07/03/2026, 8:37 AMstream_output=True on run_shell_script does not mean "send stderr into Prefect run logs."
What it actually does in Prefect 3.x is:
- capture stdout and stderr internally
- return them in the step result
- additionally mirror them to sys.stdout / sys.stderr
So stderr is not omitted entirely — it's just not routed through Prefect's logger, which is why you may not see it in the UI run log.
What happens under the hood
- stdout is read from the subprocess stream
- stderr is read from the subprocess stream
- both are stored in memory and returned from the step as:
- stdout
- stderr
- when stream_output=True, Prefect also writes:
- stdout to sys.stdout
- stderr to sys.stderr
That means stderr usually shows up in the worker/process console, not necessarily in Prefect's structured logs.
Why it doesn't show in the run log
Prefect run logs are based on log records, not arbitrary writes to sys.stderr.
run_shell_script streams stderr directly to the process stderr, instead of emitting it with a Prefect logger.
So the behavior is roughly:
- captured in step result? yes
- printed to terminal/console when stream_output=True? yes
- visible as Prefect log lines in the UI? not reliably / generally no
What you can do instead
If you need stderr content visible in Prefect logs, log it explicitly from a later step or from Python code.
For example, in deployment steps you can reference the prior step output:
yaml
pull:
- prefect.deployments.steps.run_shell_script:
id: my_script
script: bash -c 'echo hello; >&2 echo error-msg'
stream_output: true
- prefect.deployments.steps.run_shell_script:
script: |
echo "stderr was: {{ my_script.stderr }}"
Or if you're doing this in Python, read the returned stderr and log it with Prefect's logger.
Important caveat
If the shell script exits non-zero, Prefect raises before a later step can log the captured stderr unless you handle that flow differently.
Relevant source-backed conclusion
This is expected behavior based on the current implementation of `prefect.deployments.steps.run_shell_script`: stderr is captured and optionally mirrored to sys.stderr, but not emitted as Prefect log records.
If you want, I can also show you the exact return shape of run_shell_script or help with a pattern to make stderr always appear in deployment logs.