Utkarsh Jain
08/23/2025, 9:38 AMMarvin
08/23/2025, 9:38 AMMarvin
08/23/2025, 9:42 AMConnectionComponents
and feed each part from environment variables.
Here’s a complete example that:
- Builds a SqlAlchemyConnector
from separate env vars
- Saves it as a block
- Uses it in a flow
import os
from pydantic import SecretStr
from prefect import flow
from prefect_sqlalchemy.database import SqlAlchemyConnector
from prefect_sqlalchemy.credentials import ConnectionComponents
# Build the connector from discrete env vars
connector = SqlAlchemyConnector(
connection_info=ConnectionComponents(
# Choose the driver that matches what you have installed:
# - "postgresql+psycopg" for psycopg3
# - "postgresql+psycopg2" for psycopg2
driver=os.getenv("PGDRIVER", "postgresql+psycopg"),
host=os.environ["PGHOST"],
port=int(os.getenv("PGPORT", "5432")),
username=os.environ["PGUSER"],
password=SecretStr(os.environ["PGPASSWORD"]),
database=os.environ["PGDATABASE"],
# Optional extra params like SSL settings
query={"sslmode": os.getenv("PGSSLMODE", "prefer")},
),
# Optional: driver-specific connect args (e.g., for SSL certs)
# connect_args={"sslrootcert": os.getenv("PGSSLROOTCERT")},
)
# Save as a Prefect block for reuse (optional)
connector.save("pg-connector", overwrite=True)
@flow
def demo():
# Load and use the connector in your flow
with SqlAlchemyConnector.load("pg-connector") as db:
db.execute("CREATE TABLE IF NOT EXISTS demo (id INT)")
db.execute("INSERT INTO demo (id) VALUES (:id)", {"id": 1})
rows = db.fetch_many("SELECT * FROM demo")
print(rows)
if __name__ == "__main__":
demo()
Notes:
- Use PGDRIVER=postgresql+psycopg
if you’re on psycopg3 (recommended). If you use psycopg2, set PGDRIVER=postgresql+psycopg2
. Make sure the corresponding package is installed.
- You can add more connection details via query
(e.g., sslmode=require
) or connect_args
if your driver needs them.
Docs:
- Prefect + SQLAlchemy collection: prefect-sqlalchemy docs
Does this match your setup? If you need to pass additional SSL parameters (certs, keys, etc.), let me know which ones and I can show where to place them.Utkarsh Jain
08/23/2025, 9:43 AMMarvin
08/23/2025, 9:43 AMMarvin
08/23/2025, 9:46 AMpostgresql+asyncpg
.
Examples
Shell env
export PREFECT_SERVER_DATABASE_DRIVER=postgresql+asyncpg
export PREFECT_SERVER_DATABASE_HOST=<http://db.example.com|db.example.com>
export PREFECT_SERVER_DATABASE_PORT=5432
export PREFECT_SERVER_DATABASE_USER=prefect
export PREFECT_SERVER_DATABASE_PASSWORD=supersecret
export PREFECT_SERVER_DATABASE_NAME=prefect
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_APPLICATION_NAME=prefect-server
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_ENABLED=true
Docker Compose (server container)
services:
prefect-server:
image: prefecthq/prefect:3-latest
command: prefect server start
environment:
PREFECT_SERVER_DATABASE_DRIVER: postgresql+asyncpg
PREFECT_SERVER_DATABASE_HOST: ${PGHOST}
PREFECT_SERVER_DATABASE_PORT: ${PGPORT}
PREFECT_SERVER_DATABASE_USER: ${PGUSER}
PREFECT_SERVER_DATABASE_PASSWORD: ${PGPASSWORD}
PREFECT_SERVER_DATABASE_NAME: ${PGDATABASE}
PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_APPLICATION_NAME: prefect-server
PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_ENABLED: "true"
Kubernetes Pod env (snippet)
env:
- name: PREFECT_SERVER_DATABASE_DRIVER
value: postgresql+asyncpg
- name: PREFECT_SERVER_DATABASE_HOST
valueFrom:
secretKeyRef: { name: pg-secrets, key: host }
- name: PREFECT_SERVER_DATABASE_PORT
value: "5432"
- name: PREFECT_SERVER_DATABASE_USER
valueFrom:
secretKeyRef: { name: pg-secrets, key: user }
- name: PREFECT_SERVER_DATABASE_PASSWORD
valueFrom:
secretKeyRef: { name: pg-secrets, key: password }
- name: PREFECT_SERVER_DATABASE_NAME
value: prefect
Docs
- Settings reference (search for “ServerDatabaseSettings” and “SQLAlchemyConnectArgsSettings”): Prefect 3 settings reference
How are you deploying Prefect Server (Docker Compose, Kubernetes/Helm, bare VM)? I can tailor an example to your setup, including TLS/SSL if you need it.Nate
08/23/2025, 12:17 PMMarvin
08/23/2025, 12:17 PMMarvin
08/23/2025, 12:19 PMdriver
, host
, port
, user
, password
, and name
in addition to connection_url
. Prefect’s settings system maps these nested fields to environment variables by uppercasing and prefixing with PREFECT_
, so you can set split env vars like PREFECT_SERVER_DATABASE_HOST
, etc.
- You can see these fields in the code here: ServerDatabaseSettings in code
- The general rules for settings/env mapping are described in the settings reference: Prefect 3 settings reference
Practically, this means you can set:
- PREFECT_SERVER_DATABASE_DRIVER
(e.g. postgresql+asyncpg
)
- PREFECT_SERVER_DATABASE_HOST
- PREFECT_SERVER_DATABASE_PORT
- PREFECT_SERVER_DATABASE_USER
- PREFECT_SERVER_DATABASE_PASSWORD
- PREFECT_SERVER_DATABASE_NAME
Optional SQLAlchemy/TLS extras exist under the nested connect-args model (e.g. PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_ENABLED
, etc.), also visible in the same file.
Notes:
- If you set PREFECT_SERVER_DATABASE_CONNECTION_URL
, that single URL will be used. The split fields are there specifically for cases where you can’t provide a full URL.
- Use postgresql+asyncpg
as the driver for Postgres.
If you try the split env vars and still see Prefect not picking them up, can you share:
- Your Prefect version (prefect version
)
- How you’re running Server (Docker Compose, K8s, etc.)
- A quick snippet of the env you’re setting
I’ll reproduce and get you an exact fix.Nate
08/23/2025, 12:52 PMUtkarsh Jain
08/24/2025, 4:26 AMBrendan Dalpe
08/25/2025, 2:03 PMBrendan Dalpe
08/25/2025, 2:20 PMUtkarsh Jain
08/25/2025, 2:56 PM