Absolutely—here are solid, up-to-date resources for self-hosting Prefect on Google Cloud. I’m assuming Prefect 3.x; if you’re on 2.x let me know.
Key self-hosted server options
- Docker Compose on a VM (Compute Engine)
-
Docker Compose guide
-
Single Docker container (dev)
-
Local CLI server (dev)
- Kubernetes on GKE (production-ready)
-
Helm chart guide
-
prefect-helm repo
- Advanced/HA patterns and hardening
-
Scale self-hosted (HA)
-
Security settings (TLS, auth, CORS)
Running flows on GCP
- Kubernetes (GKE) workers and deployments:
Run on Kubernetes
- Cloud Run (serverless) workers and push work pools:
-
Cloud Run worker guide
-
Serverless push work pools
GCP integrations (storage, data, auth)
-
prefect-gcp integration (GCS, BigQuery, Secret Manager, Vertex AI, credentials blocks)
Common GCP reference architectures
- GKE + Helm (recommended for prod)
- Prefect Server via Helm
- PostgreSQL on Cloud SQL (private IP)
- Redis on Memorystore
- Ingress + HTTPS load balancer (or NGINX Ingress)
- Workers on GKE or Cloud Run
- Compute Engine VM + Docker Compose (simpler)
- Prefect Server + Redis + (optionally) Postgres in Compose
- For production, externalize Postgres to Cloud SQL and Redis to Memorystore
Quick-start snippets
- Install Prefect Server on GKE with Helm (values.yaml configured for Cloud SQL + Memorystore):
helm repo add prefect <https://prefecthq.github.io/prefect-helm>
helm repo update
helm install prefect-server prefect/prefect-server -n prefect -f server-values.yaml
- Example DB connection env var (Cloud SQL for Postgres):
PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://USER:PASSWORD@CLOUD_SQL_HOST:5432/prefect>
Notes and tips
- Prefect Server is a long-running service; host it on GKE or a VM rather than Cloud Run.
- For server auth/TLS and reverse proxies, see
Security settings.
- For Cloud Run workers, prefer push work pools for truly serverless orchestration.
If you share your target path (GKE vs. VM), database/Redis choices (Cloud SQL/Memorystore), and how you want to run workers (GKE, Cloud Run, or VMs), I can drop a tailored Helm values.yaml, Docker Compose override, or a minimal Terraform starter for GCP networking + Cloud SQL/Memorystore.