Janos Krnak
10/03/2025, 1:14 PMPREFECT_SERVER_DATABASE_ or PREFECT_API_DATABASE_ env vars on the API and on the background services instances?Marvin
10/03/2025, 1:14 PMMarvin
10/03/2025, 1:15 PMPREFECT_SERVER_DATABASE_* for both the API and the background services. There is no PREFECT_API_DATABASE_* setting in Prefect 3.x for server-side database configuration.
Details:
- Any container/process that runs the Prefect Server (the API service and each background service instance) should receive the same PREFECT_SERVER_DATABASE_* settings so they all point to the same Postgres database.
- Common options you can set:
- PREFECT_SERVER_DATABASE_CONNECTION_URL (recommended single URL)
- Or the component fields if you prefer:
- PREFECT_SERVER_DATABASE_DRIVER (e.g., postgresql+asyncpg)
- PREFECT_SERVER_DATABASE_HOST
- PREFECT_SERVER_DATABASE_PORT
- PREFECT_SERVER_DATABASE_USER
- PREFECT_SERVER_DATABASE_PASSWORD
- PREFECT_SERVER_DATABASE_NAME
- Optional SQLAlchemy tuning/TLS (e.g., PREFECT_SERVER_DATABASE_SQLALCHEMY_ECHO, PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_CA_FILE, etc.)
Example env snippet:
PREFECT_SERVER_DATABASE_CONNECTION_URL=<postgresql+asyncpg://prefect:prefect@my-postgres:5432/prefect>
# or:
PREFECT_SERVER_DATABASE_DRIVER=postgresql+asyncpg
PREFECT_SERVER_DATABASE_HOST=my-postgres
PREFECT_SERVER_DATABASE_PORT=5432
PREFECT_SERVER_DATABASE_USER=prefect
PREFECT_SERVER_DATABASE_PASSWORD=prefect
PREFECT_SERVER_DATABASE_NAME=prefect
Docs for reference:
- Self-hosted guide: Scalable self-hosted Prefect
- Settings reference: Settings reference (search for “ServerDatabaseSettings” and PREFECT_SERVER_DATABASE_*)
If you share how you’re deploying (Docker Compose, Kubernetes/Helm), I can tailor an example manifest with the right env vars wired into both the API and services.Janos Krnak
10/03/2025, 1:47 PMMarvin
10/03/2025, 1:47 PMMarvin
10/03/2025, 1:50 PMPREFECT_SERVER_DATABASE_* env vars on all background-service replicas and API replicas.
- Also set PREFECT_API_URL (and PREFECT_API_AUTH_STRING if auth is enabled) so services can reach the API.
Docs
- Scalable self-hosted guide: Scalable self-hosted Prefect
- Settings reference (search for “ServerServices” and “ServerDatabaseSettings”): Settings reference
If you share whether you’re on Kubernetes or Docker Compose, I can sketch a concrete deployment with multiple replicas and the right env vars.Janos Krnak
10/03/2025, 1:57 PMMarvin
10/03/2025, 1:57 PMNate
10/03/2025, 2:02 PMMarvin
10/03/2025, 2:02 PMPREFECT_SERVER_EVENTS_MESSAGING_BROKER
- PREFECT_SERVER_EVENTS_MESSAGING_CACHE
- These default to in-memory; set them to the Redis modules and provide Redis connection details for your API and background services containers. Example shape:
PREFECT_SERVER_EVENTS_MESSAGING_BROKER=prefect.server.utilities.messaging.redis
PREFECT_SERVER_EVENTS_MESSAGING_CACHE=prefect.server.utilities.messaging.redis
# Provide your Redis connection details to the broker/cache (URL/TLS/etc.)
# (I can share an exact example for Docker Compose or Helm if you’d like.)
- Make sure the Prefect images have the Redis client dependency installed.
Docs
- Scalable self-hosted Prefect: Guide
- Settings reference (see ServerEventsSettings and related): Settings reference
If you share whether you’re on Docker Compose or Kubernetes, I can give you a concrete config with the right environment variables and dependencies for Redis-backed events/logs across multiple replicas.Nate
10/03/2025, 2:03 PMMarvin
10/03/2025, 2:03 PMJanos Krnak
10/03/2025, 2:03 PMMarvin
10/03/2025, 2:07 PMPREFECT_SERVER_SERVICES_*_ENABLED flags — but avoid running duplicate replicas of the same services concurrently until the audit is complete.
- Related tracking issues you may find useful:
- Healthcheck for background services: prefect#18997
- Scheduler CPU hot spot with some RRule schedules: prefect#18861
If you’d like, I can share a Docker Compose or Helm example that:
- runs N replicas of the API,
- runs 1 replica of background services,
- and uses service toggles to isolate heavier services like the Scheduler.