Fugo Takefusa
09/22/2025, 4:41 AMMarvin
09/22/2025, 4:41 AMMarvin
09/22/2025, 4:43 AMprefect.deployments.steps.run_shell_script
is a deployment step. It runs when you execute prefect deploy
(on your machine/CI), not during a flow run. stream_output: true
streams to the deploy process’s stdout/stderr, so you’ll see it in your terminal/CI logs, but it’s not attached to any flow run and therefore does not appear in the UI.
If you want this output in the Prefect UI for each run, move the shell commands into your flow using the Prefect Shell integration. Example:
from prefect import flow
from prefect_shell.commands import shell_run_command
import logging
@flow
def show_env():
# Runs during the flow and streams to Prefect logs (visible in the UI)
shell_run_command(
'echo "User: $USER"; echo "Home Directory: $HOME"',
shell="/bin/bash", # ensure a shell that expands $VAR
stream_level=<http://logging.INFO|logging.INFO> # control log level shown in UI
)
if __name__ == "__main__":
show_env()
Notes:
- Ensure the shell integration is installed where your flow runs (worker environment): pip install "prefect[shell]"
(or the prefect-shell
package).
- You can also pass env={...}
to shell_run_command
if you need to inject or override environment variables.
Related docs:
- run_shell_script deployment step
- Prefect Shell integration
- Logging in flows
Quick check: Are you trying to see this output at deploy time or attached to each flow run? If it’s the latter, the pattern above will get it into the UI.Fugo Takefusa
09/22/2025, 4:45 AMMarvin
09/22/2025, 4:45 AMMarvin
09/22/2025, 4:46 AMprefect deploy
.
prefect.deployments.steps.run_shell_script
executes at deploy time (CI/local), not during a flow run. So this will create file
in the working directory of the deploy process (or the directory
you set), and it will not be present on the worker when the flow runs unless you make it part of your runtime (commit to your repo, upload to storage, or bake into your image).
Example deployment step:
- prefect.deployments.steps.run_shell_script:
script: |
echo "value" > file
directory: ./some/path # optional; controls where the file is created
stream_output: true
If your goal is to create the file during the actual flow run (and have it available on the worker and visible in UI logs), put the command in your flow using the Shell integration or Python I/O:
from prefect import flow
from prefect_shell.commands import shell_run_command
from pathlib import Path
@flow
def make_file():
shell_run_command('echo "value" > file', shell="/bin/bash") # visible in UI logs
# or:
Path("file").write_text("value\n")
Follow-ups:
- Where do you need this file to exist: during deploy, or during each flow run?
- If it must be available at runtime, do you want to commit it to your repo, upload to storage, or bake it into your container image?
Docs:
- run_shell_script deployment step
- Prefect Shell integrationFugo Takefusa
09/22/2025, 4:47 AMMarvin
09/22/2025, 4:47 AMMarvin
09/22/2025, 4:49 AMecho "value" > file
will create file
in the worker’s current working directory (or the directory
you specify), and your flow can read it.
- The file only exists for that run in that environment; it won’t persist across runs or outside the container/VM.
Example:
pull:
- prefect.deployments.steps.run_shell_script:
script: |
echo "value" > file
# optional:
# directory: ./some/path
stream_output: true
Notes on logs:
- Output from pull steps streams to the worker/infrastructure logs. Depending on your setup, you may see these attached to the flow run logs in the UI; if not, check the worker/infrastructure logs.
- If you need the output guaranteed in the flow run logs, run the command from within the flow (e.g., with prefect_shell
) instead.
Docs:
- run_shell_script deployment step
If you share which worker type you’re using (Docker/Kubernetes/Process) and where you want the file to live, I can suggest the best directory
and persistence approach.