<@ULVA73B9P> this does not ouput logs in the prefe...
# ask-marvin
f
@Marvin this does not ouput logs in the prefect UI: - prefect.deployments.steps.run_shell_script: script: | echo "User: $USER" echo "Home Directory: $HOME" stream_output: true expand_env_vars: true
m
thought for 116.6 seconds
You’re right — those lines won’t show up in the Prefect UI.
prefect.deployments.steps.run_shell_script
is a deployment step. It runs when you execute
prefect deploy
(on your machine/CI), not during a flow run.
stream_output: true
streams to the deploy process’s stdout/stderr, so you’ll see it in your terminal/CI logs, but it’s not attached to any flow run and therefore does not appear in the UI. If you want this output in the Prefect UI for each run, move the shell commands into your flow using the Prefect Shell integration. Example:
Copy code
from prefect import flow
from prefect_shell.commands import shell_run_command
import logging

@flow
def show_env():
    # Runs during the flow and streams to Prefect logs (visible in the UI)
    shell_run_command(
        'echo "User: $USER"; echo "Home Directory: $HOME"',
        shell="/bin/bash",                 # ensure a shell that expands $VAR
        stream_level=<http://logging.INFO|logging.INFO>          # control log level shown in UI
    )

if __name__ == "__main__":
    show_env()
Notes: - Ensure the shell integration is installed where your flow runs (worker environment):
pip install "prefect[shell]"
(or the
prefect-shell
package). - You can also pass
env={...}
to
shell_run_command
if you need to inject or override environment variables. Related docs: - run_shell_script deployment step - Prefect Shell integration - Logging in flows Quick check: Are you trying to see this output at deploy time or attached to each flow run? If it’s the latter, the pattern above will get it into the UI.
f
@Marvin can you create a file during run_shell_scripts? - prefect.deployments.steps.run_shell_script: script: | echo "value" > file
m
thought for 32.5 seconds
Short answer: yes, but only on the machine where you run
prefect deploy
.
prefect.deployments.steps.run_shell_script
executes at deploy time (CI/local), not during a flow run. So this will create
file
in the working directory of the deploy process (or the
directory
you set), and it will not be present on the worker when the flow runs unless you make it part of your runtime (commit to your repo, upload to storage, or bake into your image). Example deployment step:
Copy code
- prefect.deployments.steps.run_shell_script:
    script: |
      echo "value" > file
    directory: ./some/path   # optional; controls where the file is created
    stream_output: true
If your goal is to create the file during the actual flow run (and have it available on the worker and visible in UI logs), put the command in your flow using the Shell integration or Python I/O:
Copy code
from prefect import flow
from prefect_shell.commands import shell_run_command
from pathlib import Path

@flow
def make_file():
    shell_run_command('echo "value" > file', shell="/bin/bash")  # visible in UI logs
    # or:
    Path("file").write_text("value\n")
Follow-ups: - Where do you need this file to exist: during deploy, or during each flow run? - If it must be available at runtime, do you want to commit it to your repo, upload to storage, or bake it into your container image? Docs: - run_shell_script deployment step - Prefect Shell integration
f
@Marvin what about in the pull step? pull: - prefect.deployments.steps.run_shell_script: script: | echo "value" > file
m
thought for 75.9 seconds
Yes — putting it in the pull section will create the file on the worker just before your flow code runs. Key points: - Pull steps run on the execution environment (Docker/K8s VM, etc.) at flow run time, before your flow is imported. - So
echo "value" > file
will create
file
in the worker’s current working directory (or the
directory
you specify), and your flow can read it. - The file only exists for that run in that environment; it won’t persist across runs or outside the container/VM. Example:
Copy code
pull:
  - prefect.deployments.steps.run_shell_script:
      script: |
        echo "value" > file
      # optional:
      # directory: ./some/path
      stream_output: true
Notes on logs: - Output from pull steps streams to the worker/infrastructure logs. Depending on your setup, you may see these attached to the flow run logs in the UI; if not, check the worker/infrastructure logs. - If you need the output guaranteed in the flow run logs, run the command from within the flow (e.g., with
prefect_shell
) instead. Docs: - run_shell_script deployment step If you share which worker type you’re using (Docker/Kubernetes/Process) and where you want the file to live, I can suggest the best
directory
and persistence approach.