<@ULVA73B9P> when using prefect oss deployed using...
# marvin-ai
p
@Marvin when using prefect oss deployed using helm charts, on version 3.x.x, what is expected flow of upgrading? should i upgrade first servers, then workers, then prefect version used in flow dockerfiles? also should I upgrade single patch version at a time, or can i skip? can i skip entirely and upgrade straight into newest verions? for example, what would be flow to upgrade to 3.6.26 from either 3.6.17 or 3.4.7?
m
thought for 253.7 seconds
Great questions. Here’s the recommended approach for upgrading Prefect OSS (3.x) deployed with the Helm charts, plus concrete examples for your versions. High-level rules - Compatibility: Major versions must match between server, workers, and clients. Minor/patch can differ; if the server is older than the client/worker, you’ll see a warning but things will still work. - This is enforced by the client/server version check. See Settings reference (search for
client.server_version_check_enabled
). - Skipping versions: It’s safe to jump across patch/minor versions within 3.x. The Helm chart runs DB migrations automatically and can apply multiple steps in one upgrade. - Flow images (your Dockerfiles): They act as clients. They do not need to be upgraded in lockstep, but keeping them within the same minor as your server/worker reduces warnings and avoids missing new client APIs. Recommended order (typical) 1) Backup the database (critical). 2) Upgrade the Prefect Server via Helm (this runs DB migrations). 3) Upgrade workers (can be before or after server; recommend shortly after to reduce warnings). 4) Roll your flow images to the desired Prefect version when convenient. Notes - The Helm chart handles DB migrations (including multi-step) automatically. In multi-server mode (background services split), it uses a pre-upgrade migration job to prevent race conditions. - Downgrades are not automatic; if you need to roll back across schema changes, you’ll be restoring from backup. - Always review release notes for the versions you’re jumping across and test in staging first. - Docs and resources: - Helm deployment guide - Self-hosted scaling guide - Release notes hub and v3.6 release notes - prefect-helm charts Concrete flows A) 3.6.17 → 3.6.26 (patch upgrade) - Safe to jump directly. - Steps: - Backup DB. - Bump the Prefect image tag in your Helm values for the server (and workers, when ready). - Upgrade the server release and watch the migration job and rollout. - Upgrade worker release(s) when ready. - Optionally update your flow images to 3.6.26. Example commands (adjust your release name/namespace/values files):
Copy code
# 1) Update the chart repo
helm repo update prefect

# 2) Upgrade server
helm upgrade <server-release> prefect/prefect-server \
  -n <namespace> -f server-values.yaml --wait

# 3) Check rollout / jobs
kubectl get jobs,deploy,po -n <namespace>
kubectl rollout status deploy/<api-deployment> -n <namespace>

# 4) Upgrade workers
helm upgrade <worker-release> prefect/prefect-worker \
  -n <namespace> -f worker-values.yaml --wait
B) 3.4.7 → 3.6.26 (minor + patch jump) - Also safe to jump directly; multiple migrations will run. - Extra care: - Backup DB. - Review 3.5.x and 3.6.x release notes for behavior changes. - Expect a slightly longer migration. - Steps: Same as above. Flow images (your Dockerfiles) - You can keep running flows built on 3.4.7 or 3.6.17 against a 3.6.26 server; they’ll connect and run, though older clients may log a “server is newer” warning. Update them when you want the latest client features or to eliminate version-warning noise. Operational tips - Consider temporarily pausing schedules during the upgrade window (optional). - For multi-server/HA setups, verify the migration job completes before the API rollout completes (the Helm chart will coordinate this). - If migrations take longer, increase your rollout timeouts and ensure PostgreSQL resources are adequate. If you share your Helm values (sanitized) and release names, I can suggest exact changes for bumping the image tags and the best way to monitor the migration job in your cluster.