Short answer: - You don’t need Redis to run 2 replicas of the Prefect server. - The key is to run only one copy of the background “services” (scheduler, late-runs, repossessor, etc.) and run your API/UI pods separately. Duplicate key errors typically come from multiple schedulers racing each other. Recommended pattern (Prefect 3.x): 1) Use Postgres (not SQLite) for the server DB. 2) Scale the API/UI deployment to 2 replicas but disable services on those pods. 3) Run a separate singleton Deployment that only runs Prefect services. Why this fixes “duplicate key” errors: - Those errors generally occur when two schedulers try to create the same scheduled run at the same time. By running exactly one services process, you avoid the race. How to do it with Helm - If your chart already supports a separate “services” component, enable it and keep replicas=1. If not, use either command-line flags on the API deployment to disable services and add an extra Deployment for services (via extra manifests/extraDeploy or a second Helm release). A. API/UI Deployment (2 replicas, services disabled) - Prefer using the flag

--no-services

so you don’t need to set many env vars. - Example values.yaml snippet:

server:
  replicaCount: 2
  image:
    repository: prefecthq/prefect
    tag: <your-version>   # e.g. 3.0.x
    pullPolicy: IfNotPresent

  # Run the API/UI only
  command: ["prefect"]
  args:
    - "server"
    - "start"
    - "--no-services"
    - "--host"
    - "0.0.0.0"
    # leave UI enabled (default) unless you want to disable it with --no-ui

  env:
    - name: PREFECT_API_DATABASE_CONNECTION_URL
      valueFrom:
        secretKeyRef:
          name: prefect-db
          key: connection-url

- Alternatively, instead of

--no-services

, you can disable specific services with env vars: -

PREFECT_API_SERVICES_SCHEDULER_ENABLED=false

-

PREFECT_API_SERVICES_LATE_RUNS_ENABLED=false

-

PREFECT_API_SERVICES_REPOSSESSOR_ENABLED=false

-

PREFECT_API_SERVICES_PAUSE_EXPIRATIONS_ENABLED=false

-

PREFECT_API_SERVICES_TASK_RUN_RECORDER_ENABLED=false

This is functionally similar but more verbose. B. Singleton “services” Deployment (1 replica) - Same image and DB connection; just run services only:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: prefect-services
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prefect-services
  template:
    metadata:
      labels:
        app: prefect-services
    spec:
      containers:
        - name: services
          image: prefecthq/prefect:<your-version>
          command: ["prefect"]
          args:
            - "server"
            - "services"
            - "start"
          env:
            - name: PREFECT_API_DATABASE_CONNECTION_URL
              valueFrom:
                secretKeyRef:
                  name: prefect-db
                  key: connection-url

Notes and gotchas: - Database migrations: Make sure migrations run once per upgrade. If your chart includes a migration Job/hook, keep it as a singleton so you don’t run concurrent migrations. The server has alembic locking, but a single migration job is cleaner. - Redis: Not required for Prefect server HA; the server uses Postgres. Add Redis only if you need it for other reasons (e.g., your own flows or caching), but it’s not part of avoiding duplicate key errors for the server. - Probes: The services Deployment does not expose HTTP; it’s fine to omit readiness/liveness or use a simple exec probe if you want. - Load balancing: Your API/UI pods can sit behind a Service/Ingress and be scaled independently of the services Deployment. References - Prefect Helm chart repo: prefect-helm - Prefect server CLI options (as seen in help):

prefect server start --no-services

,

prefect server services start

- Self-hosting docs: Self-hosted Prefect

- Helm deployment guidance: Server Helm If you can share: - Which Helm chart (and version) you’re using - The exact “duplicate key” error text (table/index name) I can tailor the exact values.yaml changes to your chart and confirm the env vars/flags your chart expects.