Abhishek Singh
09/24/2025, 9:38 AMMarvin
09/24/2025, 9:38 AMMarvin
09/24/2025, 9:40 AMShellOperation from the prefect-shell integration and pass a list of commands to the commands parameter. For Windows “cmd” specifically, set shell="cmd" (and optionally extension=".cmd"). Each list item becomes a line in a temporary script that Prefect runs.
Key points:
- Use commands=[...] (preferred) instead of cramming everything into one string with &&.
- On Windows cmd, failed commands do not stop the script by default. To fail-fast, either chain with && or add || exit /b 1 to each critical line.
- Set env to pass environment variables and working_dir to change directories.
- For Unix, set shell="bash" (or sh) and add set -Eeuo pipefail to stop on failures.
Example: Windows cmd with multiple commands
from prefect import flow
from prefect_shell.commands import ShellOperation
@flow
def run_cmd_script():
op = ShellOperation(
commands=[
"echo Starting",
"dir",
"where python || exit /b 1", # fail-fast if not found
"echo Done"
],
shell="cmd",
extension=".cmd", # ensures a .cmd script on Windows
working_dir="C:\\path\\to\\repo",
env={"MY_VAR": "value"},
stream_output=True # streams output to Prefect logs
)
op() # execute
if __name__ == "__main__":
run_cmd_script()
Example: Unix bash with multiple commands
from prefect import flow
from prefect_shell.commands import ShellOperation
@flow
def run_bash_script():
op = ShellOperation(
commands=[
"set -Eeuo pipefail",
"echo Starting",
"ls -la",
"command_that_may_fail",
"echo Done"
],
shell="bash",
extension=".sh",
working_dir="/path/to/repo",
env={"MY_VAR": "value"},
)
op()
if __name__ == "__main__":
run_bash_script()
Notes on chaining:
- You can also chain within a single command: commands=["step1 && step2 && step3"]. This is fine if you prefer explicit fail-fast chaining, but multiple list entries are usually more readable.
- In cmd, to ensure the script stops on error when using one-command-per-line, add || exit /b 1 to lines that must cause failure.
Docs: prefect-shell integration
If you’re indeed targeting Windows cmd, the first example is the recommended setup. Want me to adapt it to your exact commands?Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by