Here’s a high-level, battle‑tested architecture for a self‑hosted Prefect 3 Server, plus a ready‑to‑use prefect.yaml you can drop into your repo. Assumptions - Prefect 3.x - You’ll deploy the Server and run workers in your own infra - You’ll deploy flows from source (no GitHub storage block in 3.x) Common self‑hosted architectures 1) Small team / single VM (Docker Compose) - Components - Prefect Server (API + UI) container - PostgreSQL database (managed DB preferred; local container OK for test) - Optional: reverse proxy (Nginx/Caddy) for TLS and a stable URL - One or more workers (local process or Docker/Kubernetes/ECS workers) - Typical layout - A compose file that runs Prefect Server and Postgres - Reverse proxy terminates TLS and routes to the Prefect Server - Workers run wherever your compute is (same VM, another VM, or K8s) - Links - Self-hosted with Docker Compose 2) Production/HA (Kubernetes with Helm) - Components - Prefect Server (as a Deployment), backed by an external managed Postgres (RDS/Cloud SQL/Aurora, etc.) - Ingress Controller terminating TLS (ALB/Nginx/Traefik) - Worker pools per environment (e.g., a Kubernetes worker pool for prod and one for dev) - Optional: object storage for results/artifacts (S3/GCS/Azure), centralized logging/metrics - Notes - Scale the Server replicas behind your ingress - Keep Postgres managed, backed up, and monitored - Set PREFECT_API_URL for all workers and developer machines to your public/internal URL - Links - Prefect Server Helm guide What’s actually required - Prefect Server (API + UI) - PostgreSQL database - One or more workers connected to a work pool - A reverse proxy / ingress controller for TLS and a stable URL in production - No Redis is required for Prefect 3 Server Docs you’ll likely need - Server concepts: Prefect Server (concepts) - Docker Compose: Self-hosted with Docker Compose - Helm/Kubernetes: Server Helm guide - prefect.yaml schema and usage: prefect.yaml reference A minimal prefect.yaml (Prefect 3.x) - Good starting point if you have a single process worker pool running on the same machine or in a VM.

# prefect.yaml
# Run `prefect deploy` from this directory to create the deployment(s).
deployments:
  - name: my-flow
    entrypoint: flows/my_flow.py:my_flow
    description: Minimal deployment using a process worker
    work_pool:
      name: process-pool
    parameters: {}
    tags: ["dev"]

A more complete prefect.yaml - Example targeting a Kubernetes work pool with a basic cron schedule and parameters. - job_variables are specific to your worker type (this example assumes a Kubernetes worker pool).

# prefect.yaml
deployments:
  - name: my-flow-prod
    entrypoint: flows/my_flow.py:my_flow
    description: Hourly production run on Kubernetes
    tags: ["prod"]
    parameters:
      run_mode: "prod"
      retries: 2
    schedule:
      cron: "0 * * * *"
      timezone: "UTC"
    work_pool:
      name: kubernetes-pool
      job_variables:
        image: <http://ghcr.io/your-org/your-image:latest|ghcr.io/your-org/your-image:latest>
        namespace: data-platform
        serviceAccountName: prefect-worker
        # The rest of these keys depend on your worker’s schema (resources, env, etc.)
        # env:
        #   PREFECT_LOGGING_LEVEL: INFO
        # resources:
        #   limits:
        #     cpu: "1"
        #     memory: "2Gi"
        #   requests:
        #     cpu: "500m"
        #     memory: "1Gi"
    version: "1.0.0"

How to use prefect.yaml - Initialize a recipe (optional, but handy): - Run:

prefect init

to scaffold a prefect.yaml if you don’t have one

- Create/update deployments from the file: - Run:

prefect deploy

from the project root - Make sure everything that needs to talk to your Server has PREFECT_API_URL set (e.g., https://prefect.yourdomain.com/api) If you want, share a bit about your environment (VM vs Kubernetes, cloud, managed DB, expected scale), and I can tailor a compose file or Helm values and refine the prefect.yaml to match your worker type precisely.