Chalama Reddy
02/24/2026, 12:51 PMMarvin
02/24/2026, 12:51 PMMarvin
02/24/2026, 12:56 PMtarget_session_attrs, gsslib, krbsrvname) are not supported by asyncpg and can introduce connection delays as it probes/ignores unsupported options. The recommended approach is to point Prefect at a single, highly‑available endpoint (managed writer endpoint, HAProxy/pgpool/pgBouncer, etc.) rather than listing multiple hosts in the URL.
What to do instead
- Prefer a single logical endpoint that handles failover:
- Managed DB writer endpoint (e.g., RDS Multi‑AZ writer), a VIP/Load Balancer, or pgBouncer/pgpool in front of your replicas.
- Then configure Prefect with a single URL using asyncpg:
export PREFECT_SERVER_DATABASE_CONNECTION_URL="<postgresql+asyncpg://user:password@db-endpoint:5432/prefect>"
- If you must handle host failover client‑side:
- Use DNS that returns multiple A records for one hostname and point Prefect to that single hostname.
- Reduce connection establishment wait so a down host doesn’t stall the API:
export PREFECT_SERVER_DATABASE_CONNECTION_TIMEOUT=3
- Right‑size the pool so Prefect spends less time creating new connections under load:
export PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_SIZE=20
export PREFECT_SERVER_DATABASE_SQLALCHEMY_MAX_OVERFLOW=10
export PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_TIMEOUT=30
export PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_RECYCLE=1800
- If using pgBouncer (transaction mode), disable statement caching to avoid errors and extra latency:
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_STATEMENT_CACHE_SIZE=0
- TLS/mTLS: prefer Prefect’s connect args over ssl=require (libpq option). For asyncpg use:
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_ENABLED=true
# Optional:
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_CA_FILE=/path/ca.crt
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_CERT_FILE=/path/client.crt
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_KEY_FILE=/path/client.key
- Kerberos/GSSAPI: asyncpg does not support GSSAPI auth; Prefect 3.x does not support psycopg(3) as its async driver. If you require Kerberos, you’ll need to place a proxy in front of Postgres that terminates Kerberos and exposes standard user/password or IAM auth to Prefect.
Why your current URL feels slow
- The parameters you’re using (host=host1&host=host2, gsslib, krbsrvname, target_session_attrs) are libpq/psycopg features. With asyncpg they are not supported and can cause:
- Sequential connection attempts to multiple hosts until success/timeout
- Extra round‑trips while the driver ignores unknown options
- Longer API startup and connection pool warm‑up times
Docs and references
- Self‑hosted guide (HA patterns, Postgres/Redis guidance): Self-hosted Prefect (v3)
- Settings reference (all DB settings, connect args, TLS): Settings reference
- Maintenance guidance (pooling, performance): Database maintenance
If you can share a bit more:
- Are you using Kerberos/GSS to auth to Postgres?
- Is the “slow” behavior during API startup, or ongoing (e.g., latency on API requests/flow runs)?
- Are you running through pgBouncer/pgpool already?
With those details I can suggest tighter timeouts/pool sizing (or a proxy pattern) tailored to your setup.Chalama Reddy
02/24/2026, 1:27 PMMarvin
02/24/2026, 1:27 PMMarvin
02/24/2026, 1:27 PMpostgresql+asyncpg. Synchronous drivers like psycopg2 aren’t supported, and the psycopg(3) async dialect isn’t supported for the server either.
Use:
export PREFECT_SERVER_DATABASE_CONNECTION_URL="<postgresql+asyncpg://user:password@host:5432/prefect>"
If you need features typically tied to psycopg/Libpq (e.g., Kerberos/GSS), put a proxy in front of Postgres (pgBouncer/pgpool/load balancer or a managed writer endpoint) and keep Prefect pointed at that single endpoint. You can still use psycopg in your flow code for your own databases; this limitation applies only to the Prefect orchestration database.
Docs:
- Self-hosted guide: Self-hosted Prefect (v3)
- Settings reference (DB and connect args): Settings reference