Short answer: there isn’t an official “one size fits all” spec, but for your load (≈5 flow runs/min, ~50 flows, 3 workers) a modest deployment is usually plenty. Below is a conservative, production-friendly starting point you can grow from. Assumptions - Prefect 3.x, self-hosted API/UI with a dedicated Postgres - Workers run elsewhere (their compute is sized for your workloads, not the server) Recommended baseline - API/UI (Prefect Server application) - 2 vCPU - 4–8 GB RAM - 20–50 GB SSD (for container/image/cache + logs at the host level) - Optionally run 2 replicas behind a load balancer for HA once you scale - Postgres (managed service preferred: RDS/Cloud SQL/Azure PG) - 2–4 vCPU - 8–16 GB RAM - 100–200 GB fast SSD, provision sufficient IOPS (logs can be the largest driver) - Enable the pg_trgm extension (required by Prefect) - Use a current supported PG major (e.g., 14–16) - Networking - Low latency between workers, API, and Postgres (same VPC/region if possible) Why this sizing - 5 runs/min ≈ 7,200 runs/day. Prefect’s server is primarily metadata I/O bound; most compute happens in your workers. The above comfortably supports that throughput with headroom for bursts, moderate log volume, and dashboard usage. Operational tips - Start here; scale Postgres first if you see query latency or I/O wait grow. - Prefer managed Postgres (automatic backups, monitoring, easy vertical scaling). - Monitor: - API/UI: p95 request latency, CPU > 60–70% sustained, memory pressure - DB: query latency, CPU/iowait, connections, bloat/autovacuum, index hit ratio - Prefect metrics: flow-run “start time vs scheduled time” (lag), work pool queue backlog - Tuning/maintenance: - Ensure pg_trgm is installed/enabled - Keep indexes healthy; vacuum/analyze regularly (managed PG typically handles this) - If your overall volume grows (hundreds of thousands of runs retained), consider time-based partitioning for the largest tables to keep queries snappy: Partitioning guide - Retention strategy: - Most growth comes from logs. Set and enforce log retention to keep the DB lean (archive or prune logs you don’t need long-term). When to scale up - API/UI: p95 latency > 300–500 ms or CPU > 70% sustained → add 1–2 more replicas or step up vCPU/RAM. - Postgres: CPU or I/O saturation, query latency > 100–200 ms on typical queries → add CPU/IOPS/RAM or move to a bigger class. - Workload increases (more concurrent runs / more chatty flows) → increase DB IOPS and API replicas. Docs - Prefect Server (3.x) overview: Server concepts - Self-hosted deployment guide (3.x): Self-hosted guide If you can share a bit more about: - How chatty your flows are (task count per run, log volume/level) - Whether you’ll run via Docker Compose or Kubernetes - Managed vs. self-hosted Postgres, and desired HA/SLA …I can tailor these numbers further or provide a sample docker-compose with sensible limits.

Great — with self-hosted Postgres and Docker Compose, here’s what I recommend and whether you need Redis. Do you need Redis? - Short answer: not needed. - Prefect 3.x server uses Postgres as its system-of-record; there’s no built-in requirement for Redis for scheduling, orchestration, or the UI. - Redis is only useful if you intentionally choose it for things like: - Storing results via a Redis block - A separate technology in your stack (e.g., Dask/Celery using Redis) — but that’s outside Prefect Server itself - Even without replicas, you don’t need Redis for Prefect. Sizing, tailored to your setup (Compose + self-hosted Postgres) - Prefect Server (API/UI) container host: - 2 vCPU, 4–8 GB RAM - 20–50 GB SSD for images/logs - Postgres host: - 2–4 vCPU, 8–16 GB RAM - 100–200 GB fast SSD with sufficient IOPS - Make sure pg_trgm is installed/enabled - Use PG 14–16 Key settings - Database URL:

PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECTION_URL

- Optional TLS for Postgres: -

PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_CA_FILE

-

PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_CERT_FILE

-

PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_KEY_FILE

-

PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_TLS_CHECK_HOSTNAME

- Workers and clients should point to the API with

PREFECT_API_URL

(e.g., `http://prefect-server:4200/api`) Example: docker-compose.yml

services:
  postgres:
    image: postgres:15
    environment:
      POSTGRES_USER: prefect
      POSTGRES_PASSWORD: prefect
      POSTGRES_DB: prefect
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
      - ./pg-init:/docker-entrypoint-initdb.d  # will run init SQL below
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U prefect -d prefect"]
      interval: 10s
      timeout: 5s
      retries: 5

  prefect-server:
    image: prefecthq/prefect:3-latest
    depends_on:
      postgres:
        condition: service_healthy
    environment:
      # Point server at Postgres
      PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECTION_URL: <postgresql+psycopg://prefect:prefect@postgres:5432/prefect>
      # Expose the API for workers/clients
      PREFECT_API_URL: <http://prefect-server:4200/api>
      # If you front this with a reverse proxy or access via a different hostname,
      # set UI to point to the correct API URL for browsers:
      # PREFECT_UI_API_URL: <https://your.domain.example/api>
    command: >
      bash -lc "prefect server start --host 0.0.0.0 --port 4200"
    ports:
      - "4200:4200"

  # Example worker (repeat/scale as needed)
  worker-1:
    image: prefecthq/prefect:3-latest
    depends_on:
      - prefect-server
    environment:
      PREFECT_API_URL: <http://prefect-server:4200/api>
    command: >
      bash -lc "prefect worker start --pool default"

volumes:
  pgdata:

Postgres init to enable pg_trgm (pg-init/001_enable_pg_trgm.sql)

CREATE EXTENSION IF NOT EXISTS pg_trgm;

Notes and tips - You can start with one Prefect Server replica; add another and a load balancer if you want HA later. Redis still not required. - Most scale pressure shows up in Postgres first (connections, I/O, query latency). Prefer a managed PG if you can, otherwise tune autovacuum and monitor IOPS closely. - Workers should be placed close (network-wise) to the server and database. Docs - Server concepts: https://docs-3.prefect.io/v3/concepts/server - Self-hosted guide: https://docs-3.prefect.io/v3/advanced/self-hosted - Settings reference (env vars): https://docs-3.prefect.io/v3/api-ref/settings-ref If you share how your workers will connect (same Docker network, across hosts, behind a reverse proxy), I can adapt the `PREFECT_API_URL`/`PREFECT_UI_API_URL` and networking details.