<@ULVA73B9P> I'm running prefect 3.4.13. I have se...
# ask-marvin
b
@Marvin I'm running prefect 3.4.13. I have set the
PREFECT_API_KEY
environment variable, but I'm still able to make requests without passing the api key. How do I require prefect to reject api calls that don't include the api key?
m
thought for 100.7 seconds
Short answer: setting
PREFECT_API_KEY
only affects Prefect clients talking to Prefect Cloud. It does not make a self‑hosted Prefect Server enforce authentication. So yes—you can still hit your self‑hosted API without a key, because Prefect Server doesn’t validate it. What to do instead - If you’re on Prefect Cloud: the Cloud API already requires auth for workspace resources. If you’re seeing unauthenticated calls succeed, you’re likely hitting public endpoints (e.g., health) or a local Server by mistake. Share the exact endpoint if you want me to sanity‑check it. - If you’re self‑hosting Prefect Server: there’s no built‑in request auth in 3.4.13. Put the API behind an authenticating reverse proxy (Basic Auth, OAuth/OIDC via oauth2‑proxy, etc.) or restrict access via network controls. Example: NGINX Basic Auth in front of the API 1) Create users:
Copy code
htpasswd -c /etc/nginx/.htpasswd admin
2) NGINX config:
Copy code
server {
  listen 443 ssl;
  server_name <http://prefect.example.com|prefect.example.com>;

  # ssl certs...

  location / {
    auth_basic "Restricted";
    auth_basic_user_file /etc/nginx/.htpasswd;

    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_pass <http://prefect-api:4200;>  # your Prefect API service
  }
}
If you really want a “bearer-style” shared secret, you can have the proxy reject requests missing the
Authorization: Bearer <secret>
header, but note the Prefect UI won’t add that header from the browser unless you customize it—Basic Auth or OIDC is usually friendlier for the UI. Docs for reference - Security guidance (reverse proxy, TLS, CORS): Security settings - Cloud API keys (client-side usage): Cloud API keys If you can confirm whether you’re on Cloud or self-hosted Server (and which endpoint you’re calling), I can give you a tighter config. I’d say “thrilling,” but it’s auth.
b
@Marvin I'm self-hosting
m
thought for 48.9 seconds
Got it — you’re self-hosting. In Prefect 3.x, the self-hosted Server has no built‑in request authentication.
PREFECT_API_KEY
is only used by clients when talking to Prefect Cloud. The Server won’t validate it, so unauthenticated requests will still work unless you put something in front of it. How to require auth Pick one of these common patterns and put it in front of the API (and UI): - Basic Auth via reverse proxy (simple, works fine with the UI) - OIDC/OAuth2 (via oauth2-proxy, NGINX Ingress auth, Traefik ForwardAuth, Cloudflare Access) - Private network/VPN (no public ingress) - Shared secret header enforced by the proxy (for service-to-service) Example: NGINX Basic Auth 1) Create credentials:
Copy code
htpasswd -c /etc/nginx/.htpasswd admin
2) Minimal NGINX config in front of your Prefect API:
Copy code
server {
  listen 443 ssl;
  server_name <http://prefect.example.com|prefect.example.com>;

  # ssl certs...

  location / {
    auth_basic "Restricted";
    auth_basic_user_file /etc/nginx/.htpasswd;

    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_pass <http://prefect-api:4200;>  # your Prefect API service
  }
}
Example: Require a Bearer token header at the proxy
Copy code
map $http_authorization $is_auth {
  default 0;
  "~^Bearer SUPERSECRET123$" 1;
}

server {
  listen 443 ssl;
  server_name <http://prefect.example.com|prefect.example.com>;

  location / {
    if ($is_auth = 0) { return 401; }
    proxy_pass <http://prefect-api:4200;>
  }
}
Note: browsers won’t magically add that header, so if you use the UI, Basic Auth or OIDC is friendlier. References - Prefect security guidance (TLS, proxies, CORS): Security settings If you tell me how you’re running it (Docker Compose vs. Kubernetes, which proxy/ingress, and whether you want Basic Auth or OIDC), I’ll give you a drop‑in config. I’d say it’s exciting, but it’s auth.
b
@Marvin Can I set the API to use a bearer token but not the UI?
m
thought for 49.4 seconds
Short answer: not in any useful way. - The Prefect UI is a browser app that calls the same API you want to protect. If you require a Bearer token on the API, the stock UI won’t send it, so the UI breaks. - Prefect Server doesn’t natively enforce tokens; you’d do this at a reverse proxy/ingress. Viable patterns - Cookie/OIDC for humans, Bearer for automation (recommended): - Put an auth proxy (oauth2-proxy, Authelia, etc.) in front of both UI and API. - Configure it so requests are allowed if either: - there’s a valid session cookie (UI users), or - an
Authorization: Bearer <token>
is present (automation). - Most proxies can do this “OR” logic; if not, run two hostnames: - ui.example.com → cookie/OIDC - api.example.com → Bearer token - Point the UI at the API host and enable CORS appropriately. - Basic Auth for humans, Bearer for automation: - Keep API requiring Bearer. - Put Basic Auth in front of the UI and API; browsers will automatically send Basic credentials with requests the UI makes. - Scripts can use Bearer; the proxy can allow either Basic or Bearer. - Simpler operationally, but still requires proxy logic. What won’t work securely - Requiring Bearer on the API while leaving the UI completely open and expecting it to “just work.” The UI won’t attach your token, and injecting one at the proxy leaks it to anyone using the site. If you share how you’re hosting (Docker Compose vs. Kubernetes), your proxy/ingress, and whether you prefer OIDC or Basic, I’ll give you a drop‑in config. Prefect’s security notes (TLS, CORS, proxies) are here: Security settings.
b
@Marvin I'm using k8s with ingress-nginx. Can you give me annotations for an ingress to require a bearer token?
m
thought for 57.8 seconds
Yes. With ingress-nginx you can protect just the API host with a Bearer token and leave the UI host open. Two common patterns: A) Simple static Bearer token via configuration-snippet (fast, but requires snippet annotations to be allowed) - Blocks requests missing Authorization: Bearer <token> - Optionally strips the header so your Prefect API never sees the token
Copy code
apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1>
kind: Ingress
metadata:
  name: prefect-api
  annotations:
    <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
    # Require a specific Bearer token
    <http://nginx.ingress.kubernetes.io/configuration-snippet|nginx.ingress.kubernetes.io/configuration-snippet>: |
      if ($http_authorization = "") { return 401; }
      if ($http_authorization !~* "^Bearer SUPERSECRET123$") { return 401; }
      add_header WWW-Authenticate 'Bearer realm="Prefect API"' always;
      # Optional: do not forward the token to the upstream
      proxy_set_header Authorization "";
spec:
  rules:
  - host: <http://api.prefect.example.com|api.prefect.example.com>
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: prefect-api
            port:
              number: 4200
  tls:
  - hosts:
    - <http://api.prefect.example.com|api.prefect.example.com>
    secretName: api-tls
Notes: - Replace SUPERSECRET123 with your token. Be aware it’s visible to anyone who can read this Ingress; rotate accordingly. - Some clusters disable configuration-snippet/server-snippet for security. If that’s you, use option B. B) External auth (auth-request) with a tiny validator service (recommended for secrets hygiene) - Run a small service that checks the Authorization header against a token stored in a Kubernetes Secret and returns 200 or 401. - Point ingress-nginx at it with auth-url. This keeps the secret out of annotations.
Copy code
apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1>
kind: Ingress
metadata:
  name: prefect-api
  annotations:
    <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
    <http://nginx.ingress.kubernetes.io/auth-url|nginx.ingress.kubernetes.io/auth-url>: "<http://bearer-auth.default.svc.cluster.local/validate>"
    <http://nginx.ingress.kubernetes.io/auth-snippet|nginx.ingress.kubernetes.io/auth-snippet>: |
      # Pass through the incoming Authorization header to the auth service
      proxy_set_header Authorization $http_authorization;
    # Optional: do not forward Authorization to Prefect upstream
    <http://nginx.ingress.kubernetes.io/configuration-snippet|nginx.ingress.kubernetes.io/configuration-snippet>: |
      proxy_set_header Authorization "";
spec:
  rules:
  - host: <http://api.prefect.example.com|api.prefect.example.com>
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: prefect-api
            port:
              number: 4200
  tls:
  - hosts:
    - <http://api.prefect.example.com|api.prefect.example.com>
    secretName: api-tls
Then deploy a simple validator (any language) that: - Reads the expected token from a Secret (env var or mounted file) - Returns 200 if
Authorization: Bearer <token>
matches, else 401 with
WWW-Authenticate: Bearer realm="Prefect API"
Keep the UI on a separate hostname without auth, e.g.:
Copy code
apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1>
kind: Ingress
metadata:
  name: prefect-ui
  annotations:
    <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
spec:
  rules:
  - host: <http://ui.prefect.example.com|ui.prefect.example.com>
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: prefect-ui
            port:
              number: 8080
  tls:
  - hosts:
    - <http://ui.prefect.example.com|ui.prefect.example.com>
    secretName: ui-tls
Docs if you want to dig:
- ingress-nginx external auth: External authentication - configuration-snippet annotation: configuration-snippet If you share whether snippet annotations are allowed in your cluster, I can tailor this to one path or the other and give you a tiny validator Deployment/Service manifest. It’ll be… fun. In a security-hardening sort of way.
b
@Marvin can oauth2-proxy be configured to use Authorization: Bearer <token> headers for authentication?
m
thought for 48.9 seconds
Short answer: yes, but only for real JWTs. oauth2-proxy can authenticate requests that present an Authorization: Bearer <JWT> and validate that token against an OIDC/JWT issuer. It does not natively support “opaque” static bearer strings. For a shared secret, use NGINX snippet or an auth-request microservice instead. How to do it with oauth2-proxy (JWTs) - Configure oauth2-proxy with your OIDC issuer and enable “accept JWT bearer tokens”. - Point ingress-nginx’s auth-url at oauth2-proxy’s /oauth2/auth endpoint. - For API routes, don’t set auth-signin so unauthorized requests get a 401 (no browser redirect). - For UI routes, add auth-signin so users can log in interactively (cookie session). Example oauth2-proxy config (OIDC + accept bearer JWTs) - Using flags:
Copy code
--provider=oidc
--oidc-issuer-url=<https://issuer.example.com/>
--email-domain=*
--cookie-secret=<random 32B base64>
--skip-jwt-bearer-tokens=true
(With provider=oidc, JWKS is discovered from the issuer’s metadata. Add audience/claims constraints per your IdP as needed.) - Using YAML (alpha config style):
Copy code
provider: oidc
oidcIssuerURL: <https://issuer.example.com/>
emailDomains:
  - "*"
cookie:
  secret: <random 32B base64>
skipJwtBearerTokens: true
# Optionally restrict acceptable issuers/audiences explicitly:
jwtIssuers:
  - issuer: <https://issuer.example.com/>
    jwksURL: <https://issuer.example.com/.well-known/jwks.json>
    # audiences:
    #   - <api://prefect>
Ingress (API) – require Bearer JWT, no redirects
Copy code
metadata:
  annotations:
    <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
    <http://nginx.ingress.kubernetes.io/auth-url|nginx.ingress.kubernetes.io/auth-url>: "<https://auth.example.com/oauth2/auth>"
    # No auth-signin -> 401 for missing/invalid tokens
    <http://nginx.ingress.kubernetes.io/auth-response-headers|nginx.ingress.kubernetes.io/auth-response-headers>: "X-Auth-Request-User,X-Auth-Request-Email"
    # Optional: strip Authorization before upstream
    <http://nginx.ingress.kubernetes.io/configuration-snippet|nginx.ingress.kubernetes.io/configuration-snippet>: |
      proxy_set_header Authorization "";
Ingress (UI) – interactive login allowed
Copy code
metadata:
  annotations:
    <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
    <http://nginx.ingress.kubernetes.io/auth-url|nginx.ingress.kubernetes.io/auth-url>: "<https://auth.example.com/oauth2/auth>"
    <http://nginx.ingress.kubernetes.io/auth-signin|nginx.ingress.kubernetes.io/auth-signin>: "<https://auth.example.com/oauth2/start?rd=$scheme://$host$request_uri>"
    <http://nginx.ingress.kubernetes.io/auth-response-headers|nginx.ingress.kubernetes.io/auth-response-headers>: "X-Auth-Request-User,X-Auth-Request-Email"
If you actually need an opaque static token (not a JWT): - oauth2-proxy won’t validate that. Use: - an ingress-nginx configuration-snippet to match the header, or - an auth-request service that checks the token against a Kubernetes Secret. Docs - oauth2-proxy overview: oauth2-proxy configuration - JWT bearer tokens feature: JWT bearer tokens - ingress-nginx external auth: External authentication If you share your IdP (e.g., Okta, Auth0, Azure AD) and preferred audiences/claims, I can give you exact values to drop into the config.