Joao
09/12/2025, 1:53 PMMarvin
09/12/2025, 1:53 PMMarvin
09/12/2025, 2:02 PMprefect.yaml and the prefect deploy command.
- A work pool exists and a worker will be running against your server.
1) Point your CLI/worker at your server (behind NGINX)
Set the API URL everywhere you invoke Prefect (local dev machine, CI, worker environment) to your public, proxied URL:
export PREFECT_API_URL="<https://your-domain.example.com/prefect/api>"
- This is the only variable your CLI/worker needs to talk to the API.
- If you’re running the Prefect UI separately, set its backend target with:
export PREFECT_UI_API_URL="<https://your-domain.example.com/prefect/api>"
so the UI knows where to make API calls.
Docs:
- Self-hosting Prefect Server
- Security settings and environment variables
2) Example NGINX config (with subpath, websockets, and timeouts)
If you’re exposing the UI at /prefect and the API at /prefect/api, here’s a minimal NGINX config that handles reverse proxying, websockets, and SPA routing:
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 80;
server_name <http://your-domain.example.com|your-domain.example.com>;
# If using TLS, include your ssl_certificate/ssl_certificate_key and listen 443 ssl;
# Prefect API (backend at <http://prefect-api:4200/api>)
location /prefect/api/ {
proxy_pass <http://prefect-api:4200/api/;>
proxy_http_version 1.1;
# Forwarded headers for correct scheme/host detection
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
# Websockets/log streaming
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
# Timeouts for long requests/streams
proxy_read_timeout 600s;
proxy_send_timeout 600s;
}
# Prefect UI (backend at <http://prefect-ui:8080>)
location /prefect/ {
proxy_pass <http://prefect-ui:8080/;>
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
# SPA fallback so deep links work
try_files $uri /index.html;
}
}
Notes:
- If you’re not using a subpath, drop the /prefect prefix and map /api/ to the API and / to the UI instead.
- Keep proxy_http_version 1.1 and the Upgrade/Connection headers to support websockets/log streaming in the UI.
3) A minimal prefect.yaml you can deploy
This example deploys a flow from a GitHub repository and targets a work pool. The pull section is critical so your remote execution environment (e.g., Docker/Kubernetes worker) knows how to fetch the code.
name: my-project
prefect-version: ">=3.0.0"
pull:
- prefect.deployments.steps.git_clone:
repository: "<https://github.com/your-org/your-repo.git>"
branch: "main"
# access_token: "{{ env.GH_TOKEN }}" # if needed for private repos
deployments:
- name: my-deployment
entrypoint: "flows/my_flow.py:my_flow"
work_pool:
name: "my-pool"
# Optional job variables for your chosen worker type
# job_variables:
# image: "<http://ghcr.io/your-org/your-image:{{|ghcr.io/your-org/your-image:{{> env.GIT_SHA }}"
parameters:
param1: "value"
schedules:
- cron: "0 * * * *"
timezone: "UTC"
Full docs on the YAML (fields like pull, deployments, schedules, work_pool):Marvin
09/12/2025, 2:02 PMprefect work-pool create "my-pool" --type docker
Then start a worker that points at your pool. Make sure the worker environment also has PREFECT_API_URL set to the proxied URL:
export PREFECT_API_URL="<https://your-domain.example.com/prefect/api>"
prefect worker start -p "my-pool"
5) Deploy from the YAML
From your project root (where prefect.yaml lives):
- Deploy just one deployment by name:
prefect deploy --name "my-deployment"
- Or deploy all deployments listed in your YAML:
prefect deploy --all
The CLI will talk to your server at PREFECT_API_URL and create/update the deployment(s). You’ll see them in the UI, and your worker will begin picking up flow runs based on schedules or manual triggers.
Common pitfalls to watch for
- 404s in the UI when served under a subpath: ensure your NGINX routes are correct, and SPA fallback (try_files $uri /index.html) is set for the UI location.
- Missing logs/real-time updates: check the websocket headers in NGINX (`Upgrade`/`Connection`) and timeouts.
- Workers not connecting: double-check PREFECT_API_URL everywhere (CLI, worker, CI) points to the proxied external URL, not the internal container address.
Would you like me to tailor this to your exact setup?
- Are you serving Prefect under a subpath like /prefect?
- Which work pool type are you using (process, docker, kubernetes)?
- Is your code in GitHub or bundled into an image?
I can adjust the YAML and NGINX config accordingly once I have those details.