Sophia Ponte
11/01/2025, 2:34 AMMarvin
11/01/2025, 2:34 AMMarvin
11/01/2025, 2:38 AMhelm repo add prefect <https://prefecthq.github.io/prefect-helm>
helm repo update
helm show values prefect/prefect-server --version 2025.6.4170433 | less
Example values snippets
Option A (flat under serverApi):
serverApi:
# Add an initContainer (e.g., wait for a secret or DB to be ready)
initContainers:
- name: wait-for-db
image: postgres:16
command: ["sh","-c"]
args:
- |
echo "Waiting for Postgres...";
until pg_isready -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER"; do
sleep 3
done
env:
- name: DB_HOST
valueFrom:
secretKeyRef:
name: my-db-secret
key: host
- name: DB_PORT
value: "5432"
- name: DB_USER
valueFrom:
secretKeyRef:
name: my-db-secret
key: username
# (Optional) extra volumes if you need to mount secrets/files
extraVolumes:
- name: db-secret
secret:
secretName: my-db-secret
extraVolumeMounts:
- name: db-secret
mountPath: /var/run/db
# Pick up DB URL from the ExternalSecrets-managed Secret
env:
- name: PREFECT_API_DATABASE_CONNECTION_URL
valueFrom:
secretKeyRef:
name: my-db-secret
key: url
Option B (nested under pod):
serverApi:
pod:
initContainers:
- name: wait-for-db
image: postgres:16
command: ["sh","-c"]
args:
- |
echo "Waiting for Postgres...";
until pg_isready -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER"; do
sleep 3
done
env:
- name: DB_HOST
valueFrom:
secretKeyRef:
name: my-db-secret
key: host
- name: DB_PORT
value: "5432"
- name: DB_USER
valueFrom:
secretKeyRef:
name: my-db-secret
key: username
annotations:
<http://reloader.stakater.com/auto|reloader.stakater.com/auto>: "true" # see rotation note below
env:
- name: PREFECT_API_DATABASE_CONNECTION_URL
valueFrom:
secretKeyRef:
name: my-db-secret
key: url
Keeping Prefect running when your Postgres password rotates
- Kubernetes does not update environment variables in a running pod when a Secret changes. Since Prefect Server reads the DB URL at startup, you’ll need a restart to pick up the new password.
- Recommended pattern: add an automatic restart mechanism that watches your Secret:
- Stakater Reloader operator: annotate the Deployment/pod template so pods restart when the referenced Secret changes.
- Add: <http://reloader.stakater.com/auto|reloader.stakater.com/auto>: "true"
- Reloader: https://github.com/stakater/Reloader
- With External Secrets Operator (ESO), your AWS Secret updates -> ESO refreshes the Kubernetes Secret -> Reloader restarts the Prefect API pods -> Prefect picks up the new password on startup.
Useful links
- Prefect Helm repo: github.com/PrefectHQ/prefect-helm
- Helm repo index: prefecthq.github.io/prefect-helm
- Prefect 3.x Server on Helm docs: docs-3.prefect.io/v3/advanced/server-helm
A few questions to tailor this precisely:Marvin
11/01/2025, 2:38 AMSophia Ponte
11/01/2025, 2:42 AMMarvin
11/01/2025, 2:42 AMMarvin
11/01/2025, 2:45 AMextraContainers but not an initContainers field. That means the chart doesn’t natively expose initContainers today. Extra containers run alongside the main container, so they won’t block startup like a true initContainer.
Good alternatives depending on your goal:
- If your goal is secret/password rotation (with External Secrets Operator):
- You don’t need an initContainer. Instead, ensure the API pods restart when the K8s Secret changes so they pick up the new DB URL/password at startup.
- Common approach: Stakater Reloader. Annotate the API deployment via pod annotations so it restarts on secret updates:
serverApi:
pod:
annotations:
<http://reloader.stakater.com/auto|reloader.stakater.com/auto>: "true"
Reloader: https://github.com/stakater/Reloader
- If your goal is to block API startup until Postgres is reachable:
- If your chart version exposes command/args overrides for the API container (sometimes serverApi.command / serverApi.args), you can wrap the entrypoint to wait on pg_isready before starting:
serverApi:
command: ["sh","-c"]
args:
- |
until pg_isready -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER"; do
echo "Waiting for Postgres..."; sleep 3;
done
exec prefect server start --host 0.0.0.0 --port 4200
env:
- name: DB_HOST
valueFrom:
secretKeyRef: { name: my-db-secret, key: host }
- name: DB_PORT
value: "5432"
- name: DB_USER
valueFrom:
secretKeyRef: { name: my-db-secret, key: username }
- name: PREFECT_API_DATABASE_CONNECTION_URL
valueFrom:
secretKeyRef: { name: my-db-secret, key: url }
- If command/args aren’t exposed, you can still get a true initContainer via a Helm post-renderer (e.g., kustomize/yq) to inject spec.template.spec.initContainers into the rendered Deployment:
- Render with a post-renderer script that adds:
spec:
template:
spec:
initContainers:
- name: wait-for-db
image: postgres:16
command: ["sh","-c"]
args:
- |
until pg_isready -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER"; do
echo "Waiting for Postgres..."; sleep 3;
done
env:
- name: DB_HOST
valueFrom:
secretKeyRef: { name: my-db-secret, key: host }
- name: DB_PORT
value: "5432"
- name: DB_USER
valueFrom:
secretKeyRef: { name: my-db-secret, key: username }
- Or use a policy engine (Kyverno/OPA) to mutate the pod and inject an initContainer.
To double-check what your chart version supports, you can inspect the values for your exact version (you mentioned 2025.6.4170433):
helm repo add prefect <https://prefecthq.github.io/prefect-helm>
helm repo update
helm show values prefect/prefect-server --version 2025.6.4170433 | less
Links
- Prefect Helm charts repo: github.com/PrefectHQ/prefect-helm
- Helm repo index: prefecthq.github.io/prefect-helm
- Prefect 3.x Server on Helm docs: docs-3.prefect.io/v3/advanced/server-helm
If you share the output snippet around serverApi from your helm show values, I can tailor the exact values block (annotations vs command/args) for your chart. What’s the specific behavior you want the “init” step to handle?Sophia Ponte
11/01/2025, 6:45 AMMarvin
11/01/2025, 6:45 AMMarvin
11/01/2025, 6:46 AMserverApi:
extraVolumes:
- name: pgpass
secret:
secretName: my-pgpass-secret
# 0600 in decimal is 384; YAML 1.2 may not accept 0600 octal literal
defaultMode: 384
items:
- key: pgpass
path: .pgpass
extraVolumeMounts:
- name: pgpass
mountPath: /home/prefect
readOnly: true
env:
- name: PGPASSFILE
value: /home/prefect/.pgpass
Notes:
- defaultMode: 384 is 0600. If your tooling supports it, 0o600 can also be used:
- defaultMode: 384 (recommended)
- or defaultMode: 0o600 (if your YAML parser supports it)
- If your container runs as non-root:
- A 0600 file owned by root will not be readable by a non-root user. In that case, you’ll need to write the file with the same UID/GID as the main container.
- Since the chart doesn’t expose initContainers, you can:
- Use an emptyDir + extraContainers sidecar that writes the file with umask 177 (0600) and matching runAsUser/runAsGroup, then mount that emptyDir into the main container.
- Or inject a true initContainer with a Helm post-renderer (kustomize/kyverno).
Example using a sidecar to write the file (for non-root main container):
serverApi:
pod:
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
# Source: Secret -> /src/pgpass
extraVolumes:
- name: pgpass-src
secret:
secretName: my-pgpass-secret
items:
- key: pgpass
path: pgpass
- name: pgpass
emptyDir: {}
extraContainers:
- name: write-pgpass
image: busybox:1.36
securityContext:
runAsUser: 1000
runAsGroup: 1000
command:
- sh
- -c
- |
umask 177
cp /src/pgpass /dst/.pgpass
# keep running so the file remains available
tail -f /dev/null
volumeMounts:
- name: pgpass-src
mountPath: /src
- name: pgpass
mountPath: /dst
extraVolumeMounts:
- name: pgpass
mountPath: /home/prefect
env:
- name: PGPASSFILE
value: /home/prefect/.pgpass
If this .pgpass is specifically for Prefect Server’s DB connection, you can also avoid .pgpass entirely by setting the database URL via Secret-backed env var:
serverApi:
env:
- name: PREFECT_API_DATABASE_CONNECTION_URL
valueFrom:
secretKeyRef:
name: my-db-secret
key: url
This is usually simpler and avoids file permission concerns. With External Secrets + Stakater Reloader, the pods will restart on secret changes and pick up new credentials.
Links
- Prefect Helm charts: github.com/PrefectHQ/prefect-helm
- Helm repo index: prefecthq.github.io/prefect-helm
- Reloader (for secret-driven restarts): https://github.com/stakater/Reloader
Do you know if your Prefect API container runs as root or non-root? I can tailor the snippet accordingly.