<@ULVA73B9P> which OS prefect workers support I ...
# ask-marvin
s
@Marvin which OS prefect workers support I have to run linux, MacOS and windows scripts remotely
m
thought for 752.3 seconds
Great question. Short answer: Prefect workers can run on Linux, macOS, and Windows — but which worker type you use matters for cross-OS execution. Here’s the breakdown for Prefect 3.x: - Process worker (local process) - Runs on: Linux, macOS, Windows - Best for: Running native OS scripts (bash/zsh on Linux/macOS, PowerShell/cmd on Windows) - Notes: The flow runs directly on the host that’s running the worker, so it inherits that OS and environment. - Docker/Kubernetes/ECS workers (containerized) - Host can be: Linux, macOS, Windows (with Docker Desktop/WSL2 on Windows) - Workload OS: Typically Linux (official Prefect images are Linux-based) - Windows containers: Possible only if you build your own Windows container image and run on Windows Docker/ECS with Windows containers; this is not an out-of-the-box path in Prefect. - macOS containers: Not supported by Docker; macOS workloads need a macOS host with a process worker. Recommended approach for your use case (run Linux, macOS, and Windows scripts remotely): - Run one process worker per OS on machines of that OS - Route flows/deployments to the appropriate OS via separate work pools (or work queues/tags) Example: create 3 process work pools and start a worker on each machine
Copy code
# Create pools (once)
prefect work-pool create linux-pool --type process
prefect work-pool create macos-pool --type process
prefect work-pool create windows-pool --type process

# Start a worker on each OS host
# On a Linux host:
prefect worker start --pool linux-pool --type process

# On a macOS host:
prefect worker start --pool macos-pool --type process

# On a Windows host (PowerShell):
prefect worker start --pool windows-pool --type process
Then, in your deployments, target the correct pool (or queue/tag) depending on which OS the script requires. Running OS-specific shell commands in flows - Linux (bash)
Copy code
from prefect import flow
from prefect_shell.commands import shell_run_command

@flow
def run_linux():
    shell_run_command("uname -a", shell="/bin/bash")
- macOS (zsh)
Copy code
from prefect import flow
from prefect_shell.commands import shell_run_command

@flow
def run_macos():
    shell_run_command("softwareupdate -l", shell="/bin/zsh")
- Windows (PowerShell)
Copy code
from prefect import flow
from prefect_shell.commands import shell_run_command

@flow
def run_windows():
    shell_run_command("Get-ChildItem Env:", shell="powershell")
Notes and caveats - Windows: Process workers are supported, but there are occasional edge cases; keep PowerShell vs. cmd differences in mind and prefer forward-compatible paths/quoting. - If you want container isolation, keep Linux workloads on Docker/Kubernetes/ECS workers and use process workers for macOS/Windows. - A single worker cannot natively execute scripts for multiple OSes; stand up one worker per OS (or use remote execution via SSH/WinRM from within your flow if you must centralize). Quick clarification to tailor guidance: - Are you able to run a small Prefect worker on one machine per OS (Linux, macOS, Windows)? Or do you need to orchestrate all three from a single host via remote execution (SSH/WinRM)?