<@ULVA73B9P> do you have resources for hosting pre...
# ask-marvin
t
@Marvin do you have resources for hosting prefect on self hosted gcp?
m
thought for 234.6 seconds
Absolutely—here are solid, up-to-date resources for self-hosting Prefect on Google Cloud. I’m assuming Prefect 3.x; if you’re on 2.x let me know. Key self-hosted server options - Docker Compose on a VM (Compute Engine) - Docker Compose guide - Single Docker container (dev) - Local CLI server (dev) - Kubernetes on GKE (production-ready) - Helm chart guide - prefect-helm repo - Advanced/HA patterns and hardening - Scale self-hosted (HA) - Security settings (TLS, auth, CORS) Running flows on GCP - Kubernetes (GKE) workers and deployments: Run on Kubernetes - Cloud Run (serverless) workers and push work pools: - Cloud Run worker guide - Serverless push work pools GCP integrations (storage, data, auth) - prefect-gcp integration (GCS, BigQuery, Secret Manager, Vertex AI, credentials blocks) Common GCP reference architectures - GKE + Helm (recommended for prod) - Prefect Server via Helm - PostgreSQL on Cloud SQL (private IP) - Redis on Memorystore - Ingress + HTTPS load balancer (or NGINX Ingress) - Workers on GKE or Cloud Run - Compute Engine VM + Docker Compose (simpler) - Prefect Server + Redis + (optionally) Postgres in Compose - For production, externalize Postgres to Cloud SQL and Redis to Memorystore Quick-start snippets - Install Prefect Server on GKE with Helm (values.yaml configured for Cloud SQL + Memorystore):
Copy code
helm repo add prefect <https://prefecthq.github.io/prefect-helm>
helm repo update
helm install prefect-server prefect/prefect-server -n prefect -f server-values.yaml
- Example DB connection env var (Cloud SQL for Postgres):
Copy code
PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://USER:PASSWORD@CLOUD_SQL_HOST:5432/prefect>
Notes and tips - Prefect Server is a long-running service; host it on GKE or a VM rather than Cloud Run. - For server auth/TLS and reverse proxies, see Security settings. - For Cloud Run workers, prefer push work pools for truly serverless orchestration. If you share your target path (GKE vs. VM), database/Redis choices (Cloud SQL/Memorystore), and how you want to run workers (GKE, Cloud Run, or VMs), I can drop a tailored Helm values.yaml, Docker Compose override, or a minimal Terraform starter for GCP networking + Cloud SQL/Memorystore.