Joao
09/19/2025, 11:06 PMMarvin
09/19/2025, 11:06 PMMarvin
09/19/2025, 11:08 PMsslmode=require
to the URL or supply CA/cert/key via connect args.
Example (using a single connection URL — simplest):
PREFECT_SERVER_DATABASE_CONNECTION_URL=postgresql+asyncpg://<user>:<password>@<server>.<http://postgres.database.azure.com:5432/<db_name>?sslmode=require|postgres.database.azure.com:5432/<db_name>?sslmode=require>
Or with individual fields (equivalent):
PREFECT_SERVER_DATABASE_DRIVER=postgresql+asyncpg
PREFECT_SERVER_DATABASE_HOST=<server>.<http://postgres.database.azure.com|postgres.database.azure.com>
PREFECT_SERVER_DATABASE_PORT=5432
PREFECT_SERVER_DATABASE_USER=<user>
PREFECT_SERVER_DATABASE_PASSWORD=<password>
PREFECT_SERVER_DATABASE_NAME=<db_name>
Optional TLS files (if you need to pin CA/cert/key):
PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS__TLS__CA_FILE=/path/to/ca.crt
PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS__TLS__CERT_FILE=/path/to/client.crt
PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS__TLS__KEY_FILE=/path/to/client.key
Example container snippet:
services:
prefect-server:
image: prefecthq/prefect:3-latest
command: prefect server start
environment:
PREFECT_SERVER_DATABASE_CONNECTION_URL: <postgresql+asyncpg://prefect:<password>@<server>.postgres.database.azure.com:5432/prefect?sslmode=require>
# optionally tune the pool
PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_SIZE: "10"
PREFECT_SERVER_DATABASE_SQLALCHEMY_MAX_OVERFLOW: "20"
ports:
- "4200:4200"
If you must use Microsoft Entra (Azure AD) auth
- Azure AD for Postgres works by passing a short‑lived access token as the “password” for the DB user. Tokens expire frequently.
- Prefect Server does not expose a hook to dynamically obtain a fresh token for each new DB connection; it expects a static URL/credentials.
- Because of that, Entra token auth is not supported out of the box for Prefect’s server database. Common workarounds include:
- Use a traditional Postgres user/password specifically for the Prefect database.
- Place a proxy in front of Postgres that handles Azure AD token acquisition/refresh and presents stable credentials to clients (this is non-trivial; e.g., a custom sidecar or managed solution that injects tokens on each connection).
- Build a custom Prefect Server image and wrapper that periodically rotates credentials and restarts the server gracefully (operationally fragile and generally not recommended).
Links to Prefect docs for configuration details:
- Self-hosted overview and examples: Self-hosted (Docker Compose) guide
- Settings reference: Settings reference
- Server database settings model (fields/environment variables): Server database config
If Entra is a hard requirement, let me know:
- Which identity you plan to use (Managed Identity vs Service Principal vs User)?Marvin
09/19/2025, 11:08 PMJoao
09/20/2025, 9:56 AMMarvin
09/20/2025, 9:56 AMMarvin
09/20/2025, 9:58 AMfrom prefect import flow, task
from azure.identity import DefaultAzureCredential
from azure.storage.blob import BlobServiceClient
@task
def list_containers(account_url: str):
cred = DefaultAzureCredential() # picks up MI/Workload Identity automatically
client = BlobServiceClient(account_url=account_url, credential=cred)
return [c.name for c in client.list_containers()]
@flow
def demo():
containers = list_containers("https://<storage-account>.<http://blob.core.windows.net|blob.core.windows.net>")
print(containers)
if __name__ == "__main__":
demo()
If you confirm OIDC vs SAML and any claim/group requirements, I can send a precise, tenant-ready checklist. Additionally, if you’ll be on AKS or Azure VMs, I can outline the exact identity wiring for your worker type.Joao
09/20/2025, 10:02 AMMarvin
09/20/2025, 10:03 AMMarvin
09/20/2025, 10:08 AMPREFECT_SERVER_DATABASE_CONNECTION_URL=postgresql+asyncpg://<username>:<password>@<server-name>.<http://postgres.database.azure.com:5432/<database>?sslmode=require|postgres.database.azure.com:5432/<database>?sslmode=require>
Notes:
- Flexible Server typically uses just <username>
(not username@server
like Single Server did).
- If your Azure instance requires a root CA, mount the cert in your container and add sslrootcert=/path/to/DigiCertGlobalRootG2.crt
to the URL query.
- Avoid Azure AD token-based DB authentication for the Prefect metadata DB — tokens expire and Prefect Server cannot refresh them via URL alone. Use a DB user/password.
Make sure you’re not mixing Cloud vs self-hosted auth
- Unset PREFECT_API_KEY
in self-hosted environments — that’s for Prefect Cloud.
- Use PREFECT_API_AUTH_STRING
to secure access between the UI, API, workers, and clients:
PREFECT_API_AUTH_STRING=admin:some-strong-pass
Docs:
- Self-hosted with Docker Compose
- Security settings
- Settings reference
How to meet the Entra requirement on self-hosted
Since Prefect Server doesn’t do SSO natively, front it with an auth proxy that supports Azure OIDC (oauth2-proxy is a common choice). You protect the entire Prefect UI/API port behind the proxy.
Example (docker-compose) with oauth2-proxy
services:
prefect:
image: prefecthq/prefect:3-latest
environment:
PREFECT_SERVER_DATABASE_CONNECTION_URL: "postgresql+asyncpg://<username>:<password>@<server>.<http://postgres.database.azure.com:5432/<db>?sslmode=require|postgres.database.azure.com:5432/<db>?sslmode=require>"
PREFECT_API_AUTH_STRING: "admin:some-strong-pass"
# Prefect serves UI and API on 4200 by default
expose:
- "4200"
# ... any volumes, etc.
oauth2-proxy:
image: <http://quay.io/oauth2-proxy/oauth2-proxy:v7.6.0|quay.io/oauth2-proxy/oauth2-proxy:v7.6.0>
environment:
OAUTH2_PROXY_PROVIDER: "azure"
OAUTH2_PROXY_CLIENT_ID: "<entra-app-client-id>"
OAUTH2_PROXY_CLIENT_SECRET: "<entra-app-client-secret>"
OAUTH2_PROXY_COOKIE_SECRET: "<random-32-byte-base64>"
OAUTH2_PROXY_EMAIL_DOMAINS: "*"
OAUTH2_PROXY_SCOPE: "openid profile email"
# Update to your public URL
OAUTH2_PROXY_REDIRECT_URL: "<https://prefect.mycompany.com/oauth2/callback>"
OAUTH2_PROXY_UPSTREAMS: "<http://prefect:4200>"
OAUTH2_PROXY_HTTP_ADDRESS: "0.0.0.0:4180"
OAUTH2_PROXY_PASS_AUTHORIZATION_HEADER: "true"
OAUTH2_PROXY_SET_AUTHORIZATION_HEADER: "true"
OAUTH2_PROXY_SET_XAUTHREQUEST: "true"
OAUTH2_PROXY_WHITELIST_DOMAIN: ".<http://mycompany.com|mycompany.com>"
ports:
- "80:4180"
depends_on:
- prefect
- Point your DNS (e.g., prefect.mycompany.com) to the oauth2-proxy and let it handle Azure login, then forward to Prefect.
- If you’re on Kubernetes, use NGINX Ingress with external-auth or Traefik ForwardAuth similarly.
References:
- oauth2-proxy Azure providerMarvin
09/20/2025, 10:08 AM