This message was deleted.
# marvin-ai
s
This message was deleted.
m
thought for 302.8 seconds
Great question — and nice that you’re already on Prefect 3.x (3.6.6). Upgrading a self-hosted Prefect server + worker in Docker Compose is straightforward. Below is a safe, step-by-step plan plus example Compose tweaks. If you can share your current docker-compose.yml (redact secrets), I can tailor the exact changes. Assumptions - You’re on Prefect 3.x (server and worker). - You’re using Docker Compose, likely with Postgres. If you’re on SQLite or running multiple API servers, see the notes below. High-level steps 1) Plan a short maintenance window - Stop scheduling new runs and let in-flight runs finish if you want a clean cutover. 2) Back up your database (critical) - If Postgres:
Copy code
pg_dump -h <host> -U <user> -d <db> -Fc -f prefect_backup_$(date +%F).dump
Or from the Postgres container:
Copy code
docker compose exec postgres pg_dump -U <user> -d <db> -Fc -f /tmp/prefect.dump
  docker cp <stack>_postgres_1:/tmp/prefect.dump ./prefect_backup.dump
3) Update image tags - Pin to a specific version to keep upgrades explicit, e.g. 3.6.17: -
prefecthq/prefect:3.6.17
- Alternatively, use
prefecthq/prefect:3-latest
if you’re comfortable auto-tracking the latest 3.x. - Docker Hub: prefecthq/prefect images - Latest release notes: GitHub releases - Changelog diff for the latest: 3.6.16 → 3.6.17 4) Handle database migrations - By default, the API will auto-run DB migrations on startup: -
PREFECT_API_DATABASE_MIGRATE_ON_START=true
(default) - For single-server setups, you can keep the default and just restart. - For multi-API-server or stricter production change-control, disable auto-migrate and run the migration once, explicitly:
Copy code
PREFECT_API_DATABASE_MIGRATE_ON_START=false
  docker run --rm \
    -e PREFECT_API_DATABASE_CONNECTION_URL="<postgresql+asyncpg://user:password@host:5432/prefect>" \
    prefecthq/prefect:3.6.17 \
    prefect server database upgrade -y
- Docs: - Self-hosted guide: Database migrations (self-hosted) - Settings reference: migrate_on_start setting 5) Restart services
Copy code
docker compose pull
docker compose down
docker compose up -d
6) Verify the upgrade - Server health:
Copy code
curl <http://localhost:4200/api/health>
- API version:
Copy code
curl <http://localhost:4200/api/server/version>
- Check pending migrations (should show none after upgrade):
Copy code
docker compose exec <prefect-server-service> prefect server database upgrade --dry-run
- Worker is connected:
Copy code
docker compose logs <prefect-worker-service> | tail -n 50
  docker compose exec <prefect-worker-service> prefect worker inspect
- On the new version you can also use:
Copy code
docker compose exec <prefect-server-service> prefect server status
Minimal Compose changes (single API server, Postgres) Only show the bits you’re most likely to change — image tags and the DB URL. Keep migrate_on_start as default (true) for single-server. ``` services: postgres: image: postgres:14 environment: POSTGRES_USER: prefect POSTGRES_PASSWORD: prefect POSTGRES_DB: prefect volumes: - postgres_data:/var/lib/postgresql/data prefect-server: image: prefecthq/prefect:3.6.17 # was 3.6.6 environment: PREFECT_API_DATABASE_CONNECTION_URL: postgresql+asyncpg://prefect:prefect@postgres:5432/prefect # migrate_on_start defaults to true; you can omit it command: prefect server start --no-services ports: - "4200:4200" depends_on: - postgres prefect-services: image: prefecthq/prefect:3.6.17 environment: PREFECT_API_DATABASE_CONNECTION_URL: postgresql+asyncpg://prefect:prefect@postgres:5432/prefect
command: prefect server services start depends_on: - prefect-server prefect-worker: image: prefecthq/prefect:3.6.17 environment: PREFECT_API_URL: http://prefect-server:4200/api command: prefect worker start --pool <your-pool-name> depends_on: - prefect-server volumes: postgres_data:
Copy code
If you prefer manual migrations (e.g., multi-server)
Add a one-off migration service and disable auto-migrate on API servers.
services: postgres: image: postgres:14 environment: POSTGRES_USER: prefect POSTGRES_PASSWORD: prefect POSTGRES_DB: prefect prefect-migrator: image: prefecthq/prefect:3.6.17 environment: PREFECT_API_DATABASE_CONNECTION_URL: postgresql+asyncpg://prefect:prefect@postgres:5432/prefect command: prefect server database upgrade -y restart: "no" depends_on: - postgres prefect-server: image: prefecthq/prefect:3.6.17 environment: PREFECT_API_DATABASE_CONNECTION_URL: postgresql+asyncpg://prefect:prefect@postgres:5432/prefect PREFECT_API_DATABASE_MIGRATE_ON_START: "false" command: prefect server start --no-services depends_on: prefect-migrator: condition: service_completed_successfully prefect-services: image: prefecthq/prefect:3.6.17 environment: PREFECT_API_DATABASE_CONNECTION_URL: postgresql+asyncpg://prefect:prefect@postgres:5432/prefect command: prefect server services start depends_on: prefect-migrator: condition: service_completed_successfully prefect-worker: image: prefecthq/prefect:3.6.17 environment: PREFECT_API_URL: http://prefect-server:4200/api command: prefect worker start --pool <your-pool-name> depends_on: - prefect-server ``` Notes and tips - Workers are stateless; upgrading them is just updating the image tag and restarting. You can do a rolling update (bring up a new worker on the new tag, then retire the old one) to avoid any polling gaps. - If you’re actually on Prefect 2.x “agents” (the old model), the migration path is different. In Prefect 3.x, we use “workers.” If you’re running
prefect agent start
, let me know and I’ll outline the 2.x → 3.x steps. - If you use SQLite, make sure you are not running multiple API servers; SQLite is fine for single-node testing but not for scaled/multi-process deployments. Helpful links - Self-hosted deployments: Self-hosted guide - Settings reference (migrate_on_start): Settings ref - Docker images: Docker Hub - Releases: GitHub releases - Latest changelog diff: 3.6.16 → 3.6.17 Want me to review your current docker-compose.yml and give you an exact patch (including your pool name and DB URL)? If so, paste it here (with secrets redacted). Also let me know: - Postgres or SQLite? - Single API server or multiple?