<@ULVA73B9P> why dont i see stderr output in the r...
# ask-marvin
a
@Marvin why dont i see stderr output in the run log for a run_shell_script deploymentstep when stream_output is set to true
m
thought for 104.5 seconds
stream_output=True
on
run_shell_script
does not mean "send stderr into Prefect run logs." What it actually does in Prefect 3.x is: - capture
stdout
and
stderr
internally - return them in the step result - additionally mirror them to
sys.stdout
/
sys.stderr
So stderr is not omitted entirely — it's just not routed through Prefect's logger, which is why you may not see it in the UI run log. What happens under the hood -
stdout
is read from the subprocess stream -
stderr
is read from the subprocess stream - both are stored in memory and returned from the step as: -
stdout
-
stderr
- when
stream_output=True
, Prefect also writes: - stdout to
sys.stdout
- stderr to
sys.stderr
That means stderr usually shows up in the worker/process console, not necessarily in Prefect's structured logs. Why it doesn't show in the run log Prefect run logs are based on log records, not arbitrary writes to
sys.stderr
.
run_shell_script
streams stderr directly to the process stderr, instead of emitting it with a Prefect logger. So the behavior is roughly: - captured in step result? yes - printed to terminal/console when
stream_output=True
?
yes - visible as Prefect log lines in the UI? not reliably / generally no What you can do instead If you need stderr content visible in Prefect logs, log it explicitly from a later step or from Python code. For example, in deployment steps you can reference the prior step output:
Copy code
yaml
pull:
  - prefect.deployments.steps.run_shell_script:
      id: my_script
      script: bash -c 'echo hello; >&2 echo error-msg'
      stream_output: true

  - prefect.deployments.steps.run_shell_script:
      script: |
        echo "stderr was: {{ my_script.stderr }}"
Or if you're doing this in Python, read the returned
stderr
and log it with Prefect's logger. Important caveat If the shell script exits non-zero, Prefect raises before a later step can log the captured stderr unless you handle that flow differently. Relevant source-backed conclusion This is expected behavior based on the current implementation of `prefect.deployments.steps.run_shell_script`: stderr is captured and optionally mirrored to
sys.stderr
, but not emitted as Prefect log records. If you want, I can also show you the exact return shape of
run_shell_script
or help with a pattern to make stderr always appear in deployment logs.