<@ULVA73B9P> is it possible to configure proxy bet...
# ask-marvin
y
@Marvin is it possible to configure proxy between a prefect worker and a server
m
thought for 163.0 seconds
Yes. Prefect workers/clients (Prefect 3.x) use httpx under the hood and automatically respect standard proxy environment variables. You don’t need a Prefect-specific proxy setting—just set the usual env vars before starting your worker. Quick setup:
Copy code
# Required if you’re behind a corporate proxy
export HTTPS_PROXY=<http://proxy.mycorp.com:8080>
# Optional: exclude internal hosts or your self-hosted server
export NO_PROXY=localhost,127.0.0.1,*.mycorp.local,prefect.internal

# If your proxy/corporate CA uses custom certificates
export SSL_CERT_FILE=/path/to/company-ca-bundle.pem
# Or Prefect-specific override
export PREFECT_API_SSL_CERT_FILE=/path/to/company-ca-bundle.pem
# As a last resort (dev only):
export PREFECT_API_TLS_INSECURE_SKIP_VERIFY=true
Then run your worker as usual:
Copy code
prefect worker start -p <your-work-pool>
Notes: - Works for both Prefect Cloud and self-hosted Server. Just make sure
PREFECT_API_URL
(and
PREFECT_API_KEY
for Cloud) are set. - WebSocket connections (used for logs/events) also honor these proxy variables. If your proxy blocks WSS/CONNECT, you may need to allowlist Prefect endpoints or add them to
NO_PROXY
. - For Kubernetes, add these env vars to the worker Deployment (and optionally to your base job template if your flow pods also need outbound proxy). Helpful links: - Configure network access for restricted environments - Troubleshoot Prefect Cloud (proxy/SSL tips) - Community discussion: Using Prefect with proxies If you can share a bit more about your setup (Cloud vs. self-hosted, OS/runner type, whether the proxy requires auth, and any domains you need to exclude), I can provide a tailored snippet (including Windows/K8s examples).
y
@Marvin So for example I have server a (of prefect server), serve b (of prefect worker), and server c a and b are usually reachable to each other but if a and b cannot connect, we need c as a jump server for communication and prefect worker is run using systemd, how can I configure this when I need the proxyjump
m
thought for 54.4 seconds
Short answer: Prefect doesn’t support SSH ProxyJump directly because workers talk to the API over HTTPS/WSS, not SSH. To use a “jump” host, run traffic through a proxy reachable on server C and point the worker on server B to it via env vars. Two common patterns: 1) HTTP/HTTPS forward proxy on C (recommended if you control C) - Run squid/tinyproxy/HAProxy on C and allow B to reach it - Set
HTTPS_PROXY
on B and start your worker Example systemd unit on B:
Copy code
# /etc/systemd/system/prefect-worker.service
[Unit]
Description=Prefect Worker
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=prefect
# Prefect Server API URL
Environment=PREFECT_API_URL=<https://a.example.com/api>
# If using Prefect Cloud, also set PREFECT_API_KEY=...
# Use the proxy on C
Environment=HTTPS_PROXY=<http://c.example.com:3128>
# Avoid proxy for local/internal if needed
Environment=NO_PROXY=localhost,127.0.0.1,<http://a.example.com|a.example.com>
# Custom CA if your proxy rewrites TLS
Environment=SSL_CERT_FILE=/etc/ssl/certs/company-ca.pem
ExecStart=/usr/local/bin/prefect worker start -p my-work-pool
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
2) SSH “jump” via a SOCKS5 tunnel on B → C - Create a persistent SOCKS proxy using ssh or autossh - Point Prefect to it with
HTTPS_PROXY=<socks5h://127.0.0.1:1080>
- Important: httpx needs SOCKS support; ensure the worker’s Python env has
httpx[socks]
installed Systemd units (B): SOCKS tunnel:
Copy code
# /etc/systemd/system/socks-tunnel@.service
[Unit]
Description=SOCKS5 tunnel via %i
After=network-online.target
Wants=network-online.target

[Service]
User=prefect
Environment=AUTOSSH_GATETIME=0
ExecStart=/usr/bin/autossh -M 0 -N \
  -o ServerAliveInterval=30 -o ServerAliveCountMax=3 \
  -D 127.0.0.1:1080 prefect@%i
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
Prefect worker (via SOCKS):
Copy code
# /etc/systemd/system/prefect-worker-socks.service
[Unit]
Description=Prefect Worker (via SOCKS proxy)
After=socks-tunnel@c.example.com.service
Wants=socks-tunnel@c.example.com.service

[Service]
User=prefect
Environment=PREFECT_API_URL=<https://a.example.com/api>
# Proxy through local SOCKS (tunneled via C). Requires httpx[socks].
Environment=HTTPS_PROXY=<socks5h://127.0.0.1:1080>
Environment=NO_PROXY=localhost,127.0.0.1,<http://a.example.com|a.example.com>
Environment=SSL_CERT_FILE=/etc/ssl/certs/company-ca.pem
ExecStart=/usr/local/bin/prefect worker start -p my-work-pool
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
Then:
Copy code
# One-time
sudo apt-get install autossh  # or your package manager
pip install "httpx[socks]"    # in the same env the worker uses

# Start
sudo systemctl enable --now <mailto:socks-tunnel@c.example.com|socks-tunnel@c.example.com>
sudo systemctl enable --now prefect-worker-socks
About “automatic” fallback when A↔️B fails - Env vars don’t provide built-in proxy fallback. Easiest ops approach is to: - Keep two services (direct and via proxy) and switch when needed, or - Always route via the proxy (simple and reliable), or - Use separate tooling (HAProxy on B, or a wrapper script) to detect connectivity and choose the path before launching the worker. - If you want me to, I can share a small wrapper script + systemd unit that tries direct first, then starts the SOCKS service and exports
HTTPS_PROXY
if direct health-check fails. Notes - WebSockets (used by Prefect for logs/events) will work through HTTP proxies that allow CONNECT and through SOCKS proxies. - If your proxy inserts a corporate CA, set
SSL_CERT_FILE
(or
PREFECT_API_SSL_CERT_FILE
), or in a pinch
PREFECT_API_TLS_INSECURE_SKIP_VERIFY=true
for testing only. - For Kubernetes workers, the same env vars go into the Deployment/Pod spec instead of systemd. Docs and references - Configure network access in restricted environments: Prefect docs
- Troubleshooting (proxy/SSL tips): Prefect Cloud guide - Community thread on proxies: GitHub Discussion If you can confirm: - Whether C can run a standard HTTP proxy or you must use SSH - Whether you want permanent proxying or automatic failover - Your OS/distro on B and how Prefect is installed (system Python vs venv) …I can tailor the exact units/commands for your setup.