Madushan Pathirana
09/17/2025, 11:18 AMMarvin
09/17/2025, 11:18 AMMarvin
09/17/2025, 11:24 AMPREFECT_SERVER_API_AUTH_STRING="username:password"
- Clients/workers env var: PREFECT_API_AUTH_STRING="username:password"
- Do not set PREFECT_API_KEY
when using self‑hosted Server (that’s for Prefect Cloud).
- Useful server security settings:
- Bind to localhost (and proxy from there): PREFECT_SERVER_API_HOST=127.0.0.1
- CORS: set only your UI origin, not *
- PREFECT_SERVER_API_CORS_ALLOWED_ORIGINS="<https://prefect.example.com>"
- Optional base path when proxying under a subpath:
- PREFECT_SERVER_API_BASE_PATH="/prefect"
- Docs: Security settings
Adding a real “user authentication UI”
Because Prefect Server doesn’t have user accounts or SSO built‑in, the standard approach is to put it behind a reverse proxy that provides login (OIDC/SAML) and then forwards requests to Prefect:
Option A (most common): oauth2‑proxy + NGINX/Traefik
- oauth2‑proxy handles the login UI (Google Workspace, Okta, Azure AD, etc.).
- The proxy:
- Requires user login for the UI/API routes
- After a user is authenticated, injects the Prefect Basic Auth header when proxying to the API (so the API’s auth_string
is satisfied automatically)
- High‑level flow:
- Browser → NGINX (TLS, OIDC via oauth2‑proxy) → add Authorization: Basic base64(user:pass)
→ Prefect API
- Workers/CLI use PREFECT_API_AUTH_STRING
directly to call API (no login)
- oauth2‑proxy docs: https://oauth2-proxy.github.io/oauth2-proxy/
Minimal NGINX idea (conceptual)
# TLS + OIDC handled via oauth2-proxy/forward_auth
location / {
auth_request /oauth2/auth;
error_page 401 = /oauth2/sign_in;
# Inject Prefect API basic auth toward upstream after user is logged in
proxy_set_header Authorization "Basic <base64-of-username:password>";
proxy_pass <http://127.0.0.1:4200/;> # or /prefect if you set BASE_PATH
}
Notes:
- Keep the username:password
identical to PREFECT_SERVER_API_AUTH_STRING
so both human and machine clients are authorized.
- Traefik and Envoy can do the same with forwardAuth and header‑injection.
Option B: AWS ALB authentication (OIDC/Cognito)
- ALB can front the UI/API with OIDC auth. If you go this route, it’s simpler to disable Prefect’s auth_string
and let ALB do auth for humans.
- If you still want machine clients (workers) to connect non‑interactively, consider:
- Keeping a private, internal listener or bypass path for workers, or
- Not using ALB auth and sticking to the oauth2‑proxy pattern above where workers just send the Basic header.
- ALB OIDC auth docs: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/listener-authenticate-users.html
Network hardening (fits your single‑EC2 in ap-southeast-1)
- Run Prefect Server only on loopback and front it with a proxy:
- PREFECT_SERVER_API_HOST=127.0.0.1
- Start:
export PREFECT_SERVER_API_AUTH_STRING='prefect:<long-random-secret>'
export PREFECT_SERVER_API_HOST=127.0.0.1
export PREFECT_SERVER_API_PORT=4200
export PREFECT_SERVER_API_CORS_ALLOWED_ORIGINS='<https://prefect.example.com>'
prefect server start --ui
- TLS: terminate at NGINX/Traefik or an ALB with an ACM cert. Redirect HTTP→HTTPS.
- Private vs public access:Marvin
09/17/2025, 11:24 AMexport PREFECT_SERVER_DATABASE_CONNECTION_URL='<postgresql+psycopg://user:pass@rds-host:5432/prefect>'
export PREFECT_SERVER_DATABASE_TLS__ENABLED=true
export PREFECT_SERVER_DATABASE_TLS__CA_FILE=/etc/ssl/certs/rds-ca.pem
export PREFECT_SERVER_DATABASE_TLS__CHECK_HOSTNAME=true
- Restrict the RDS SG to only the EC2’s SG; disable public access on RDS.
What Prefect Server does NOT provide
- No multi‑user UI, no RBAC, no API keys. Those are Prefect Cloud features. If you decide you need built‑in user management and workspace‑level permissions, Cloud is the simpler path.
How this maps to your setup (single EC2 + RDS in ap-southeast-1)
- Recommended starting point:
1) Keep Prefect Server bound to 127.0.0.1; use oauth2‑proxy + NGINX on the same EC2 to provide SSO login and TLS.
2) Set PREFECT_SERVER_API_AUTH_STRING
and have NGINX inject the Basic header upstream for authenticated users.
3) Configure workers/flows on the EC2 to use PREFECT_API_URL=<http://127.0.0.1:4200/api>
and PREFECT_API_AUTH_STRING
so they connect locally without going through SSO.
4) Secure RDS with TLS and SG rules.
5) Decide public vs private:
- Private: no public IP; access UI via VPN/SSM.
- Public: ALB or public NGINX 443, restricted SG + WAF, mandatory SSO.
Relevant docs and references
- Prefect security settings: Security settings
- Prefect Server settings (incl. base path, ports via the Helm doc but same settings apply): Server settings reference
- oauth2‑proxy: oauth2-proxy
- AWS ALB OIDC auth: ALB authenticate users
A few questions so I can tailor a concrete config for you:
- Which identity provider do you want to use (Google, Okta, Azure AD, etc.)?
- Do you want the UI publicly reachable, or should it remain private (VPN/SSM only)?
- Will any workers run off‑box later, or will everything stay on that single EC2?
- Do you want the UI/API under a subpath (e.g., https://example.com/prefect) or its own subdomain?Madushan Pathirana
09/17/2025, 3:26 PMMarvin
09/17/2025, 3:26 PMMarvin
09/17/2025, 3:28 PMserver.api.auth_string
, the UI asks once and then caches the credential in the browser so subsequent page loads don’t prompt again.
Ways to enforce re-authentication
- Use a reverse proxy with real login and short-lived/session cookies (recommended)
- Put the UI/API behind oauth2-proxy (or Traefik forward auth / NGINX OIDC / AWS ALB auth).
- Configure it for session-only cookies (expire on browser close) or short TTL/refresh so users must re-authenticate regularly.
- The proxy can inject the Prefect Basic header upstream so your API’s auth_string
is still satisfied without users seeing “admin:pass”.
- Example idea with oauth2-proxy (session cookie):
oauth2-proxy \
--provider=oidc \
--oidc-issuer-url=<your-idp> \
--client-id=<id> \
--client-secret=<secret> \
--cookie-secret=<32-byte-base64> \
--cookie-secure=true \
--cookie-expire=0 # session cookie, requires login next browser session
--cookie-refresh=10m # re-validate periodically
- NGINX forwards to Prefect and injects Basic auth after a user is authenticated:
location / {
auth_request /oauth2/auth;
error_page 401 = /oauth2/sign_in;
proxy_set_header Authorization "Basic <base64-of-username:password>";
proxy_pass <http://127.0.0.1:4200/;>
}
- Manual “logout” now
- There is no built-in “logout” button in the self-hosted UI.
- To force the prompt again immediately, clear site data for your Prefect UI origin (browser settings or DevTools → Application → Local Storage/IndexedDB → clear). Incognito/private windows will also prompt each time.
Why this is necessary
- The auth_string
gate is a single shared Basic credential. The UI caches it client-side; there’s no server-side session or configurable “always prompt” toggle.
- For multi-user login behavior, you need to front the UI/API with an auth layer (SSO proxy or ALB auth) and control session policy there.
If you want, I can share a concrete NGINX + oauth2-proxy config tailored to your single-EC2-in-Singapore setup. Are you open to SSO via Google/Okta/Azure AD, and do you want the UI public or private (VPN/SSM only)?
Refs:
- Prefect security settings: Security settings
- Server settings reference: Server settings reference
- oauth2-proxy: oauth2-proxy
- AWS ALB OIDC auth: ALB authenticate usersMadushan Pathirana
09/17/2025, 6:07 PMMarvin
09/17/2025, 6:07 PMMarvin
09/17/2025, 6:09 PMprefect config view
You should see:
PREFECT_SERVER_API_AUTH_STRING='****'
If it’s missing, the env var isn’t reaching the process that runs prefect server start
.
2) Make sure you’re not accidentally pointing the UI to Prefect Cloud
- If PREFECT_API_URL
is set in the container to a Cloud URL, the UI will call Cloud (and won’t prompt).
- For self-hosted Server, either unset PREFECT_API_URL
or set it explicitly to your Server API (e.g., `https://your-alb.example.com/api`).
3) Test the API directly to confirm auth is required
- Without auth (should be 401 with WWW-Authenticate):
curl -i <https://your-alb.example.com/api/health>
- With auth (should be 200):
curl -i -u admin:pass <https://your-alb.example.com/api/health>
If you get 200 without auth, the server is not enforcing auth_string
. If you get 401 but no WWW-Authenticate: Basic
header, the proxy/ALB is stripping it and the browser won’t show a prompt.
4) Confirm your ALB/proxy is passing headers and not rewriting paths
- If you’re routing under a subpath, set:
PREFECT_SERVER_API_BASE_PATH="/prefect"
and ensure the ALB forwards that path to the container unchanged.
- Make sure the ALB is not removing the WWW-Authenticate
header from 401 responses.
5) Try a private/incognito window
- Browsers cache Basic credentials per-origin. If you entered them previously, you won’t be prompted again.
6) Health checks and ALB
- If you enable API auth, ALB health checks to /api/health
will return 401. Point the health check to /
(UI static) or another public path so targets stay healthy.
Known-good minimal server env (example)
PREFECT_SERVER_API_AUTH_STRING=admin:pass
PREFECT_SERVER_API_HOST=0.0.0.0
PREFECT_SERVER_API_PORT=4200
# Optional: if you proxy under a subpath
# PREFECT_SERVER_API_BASE_PATH=/prefect
# Avoid pointing to Cloud
# unset PREFECT_API_URL
Run:
prefect server start --ui
If you want a first-class login experience (per-user auth, repeated prompts, short-lived sessions), put the UI/API behind oauth2-proxy (OIDC/SAML) or ALB authentication and let the proxy manage sessions, then inject the Authorization: Basic <base64(admin:pass)>
header upstream to Prefect. Details here: Security settings.
If you can share:
- The Prefect version in your image (prefect version
)
- The exact command your ECS task runs (entrypoint/command)
- Whether PREFECT_API_URL
is set
- Your ALB listener/rule setup (path-based routing, any auth action)
…I can pinpoint why the prompt isn’t appearing and provide a targeted fix.Madushan Pathirana
09/17/2025, 6:25 PMMarvin
09/17/2025, 6:26 PMMarvin
09/17/2025, 6:27 PMprefect config view
You should see:
PREFECT_SERVER_API_AUTH_STRING='****'
If it’s missing, the env var didn’t reach the process that runs prefect server start
.
2) From your workstation, test the ALB endpoint
- Expect 401 with WWW-Authenticate when no credentials:
curl -i https://<your-alb>/api/health
- Expect 200 with credentials:
curl -i -u admin:pass https://<your-alb>/api/health
Interpretation:
- 200 without creds → server is not enforcing auth (auth_string not applied).
- 401 without a WWW-Authenticate: Basic header → something in front (ALB/other proxy) is stripping the header so the browser won’t show a prompt.
3) Common fixes
- Ensure your task is actually starting Prefect with:
prefect server start --ui
- Make sure nothing sets PREFECT_API_KEY in that container (that’s for Cloud and can confuse things).
- Bind to all interfaces for ALB traffic:
PREFECT_SERVER_API_HOST=0.0.0.0
PREFECT_SERVER_API_PORT=4200
- If using a subpath behind ALB, set:
PREFECT_SERVER_API_BASE_PATH=/prefect
and make sure ALB forwards the path unchanged.
- Try an incognito window to rule out cached Basic creds.
4) ALB health checks
- If auth is enabled, ALB health checks to /api/health will get 401. Point health checks to “/” (the static UI) or another unauthenticated path.
If you share:
- The output of prefect config view
from inside the ECS task
- The result of the two curl commands above
- Your ECS task command/entrypoint and ALB listener/rule setup
…I can zero in on the exact cause and give you a concrete fix.
Reference: Prefect security settings