Shubham
11/06/2025, 9:18 AMMarvin
11/06/2025, 9:18 AMMarvin
11/06/2025, 9:30 AM# Create pools (once)
prefect work-pool create linux-pool --type process
prefect work-pool create macos-pool --type process
prefect work-pool create windows-pool --type process
# Start a worker on each OS host
# On a Linux host:
prefect worker start --pool linux-pool --type process
# On a macOS host:
prefect worker start --pool macos-pool --type process
# On a Windows host (PowerShell):
prefect worker start --pool windows-pool --type process
Then, in your deployments, target the correct pool (or queue/tag) depending on which OS the script requires.
Running OS-specific shell commands in flows
- Linux (bash)
from prefect import flow
from prefect_shell.commands import shell_run_command
@flow
def run_linux():
shell_run_command("uname -a", shell="/bin/bash")
- macOS (zsh)
from prefect import flow
from prefect_shell.commands import shell_run_command
@flow
def run_macos():
shell_run_command("softwareupdate -l", shell="/bin/zsh")
- Windows (PowerShell)
from prefect import flow
from prefect_shell.commands import shell_run_command
@flow
def run_windows():
shell_run_command("Get-ChildItem Env:", shell="powershell")
Notes and caveats
- Windows: Process workers are supported, but there are occasional edge cases; keep PowerShell vs. cmd differences in mind and prefer forward-compatible paths/quoting.
- If you want container isolation, keep Linux workloads on Docker/Kubernetes/ECS workers and use process workers for macOS/Windows.
- A single worker cannot natively execute scripts for multiple OSes; stand up one worker per OS (or use remote execution via SSH/WinRM from within your flow if you must centralize).
Quick clarification to tailor guidance:
- Are you able to run a small Prefect worker on one machine per OS (Linux, macOS, Windows)? Or do you need to orchestrate all three from a single host via remote execution (SSH/WinRM)?