<@ULVA73B9P> i'm hosting prefect on my server, mos...
# ask-marvin
t
@Marvin i'm hosting prefect on my server, most of the time jobs run smoothly but sometime when i manually schedule a job or update some deployments, it return 503 service unavailable, i checked the logs and it said sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked [SQL: UPDATE deployment SET last_polled=:last_polled, status=:status, updated=strftime('%Y-%m-%d %H:%M:%f000', 'now') WHERE deployment.id IN (SELECT 1 FROM (SELECT 1) WHERE 1!=1) OR deployment.work_queue_id IN (:work_queue_id_1_1)]
m
thought for 113.5 seconds
Ah, SQLite. Great for a quiet life, less great when multiple things poke it at once. The “database is locked” explains your intermittent 503s: when Prefect’s API tries to write and SQLite is already busy, the request times out and you get a 503. What’s happening - SQLite allows only one writer at a time. Manual scheduling and deployment updates tend to cause bursts of writes, which collide. - If you’re running more than one API process/replica, or the DB file is on a networked filesystem (NFS), this gets worse. Quick mitigations (if you must stay on SQLite) - Enable WAL mode once (persists in the DB file):
Copy code
sqlite3 /path/to/prefect.db 'PRAGMA journal_mode=WAL; PRAGMA wal_checkpoint(TRUNCATE);'
- Increase the SQLite busy timeout (gives writes more time to wait): - Use a URL with timeout, e.g.
Copy code
sqlite+aiosqlite:////path/to/prefect.db?timeout=30.0
- Ensure a single API process touches SQLite: - Do not scale the Prefect API to multiple workers/replicas against SQLite. - If you’re running uvicorn/gunicorn, set workers=1. - Put the DB on local disk (not NFS). The real fix: move to Postgres - SQLite is fine for dev, but for production concurrency you want Postgres. - Steps: 1) Provision Postgres and create a database/user. 2) Point Prefect Server at Postgres with an async URL, e.g.
Copy code
<postgresql+asyncpg://prefect:yourpass@your-host:5432/prefect>
3) Enable auto-migrations on start so the schema is created. 4) Restart the API. About settings/env vars - Prefect 3.x supports configuring the database URL and auto-migration via environment variables/settings. Depending on your exact version/build, the names are typically one of: - Database URL: PREFECT_API_DATABASE_CONNECTION_URL (sometimes packaged as PREFECT_SERVER_DATABASE_CONNECTION_URL) - Auto-migrate: PREFECT_API_DATABASE_MIGRATE_ON_START=true (sometimes PREFECT_SERVER_DATABASE_MIGRATE_ON_START) - I’ll confirm the exact variable names and examples for your setup once you tell me: - Your Prefect version (prefect version) - How you run the server (prefect server start, Docker/Compose, Helm, something custom) - Whether you have more than one API process/replica - Where your SQLite file lives (local disk vs NFS) Data migration note - There isn’t a built-in “migrate SQLite data to Postgres” tool in Prefect. If you need to preserve history, you can use external tools (e.g., pgloader) or accept a fresh DB on Postgres. If you share your setup details, I’ll give you exact commands/env vars for your path (including verified Postgres URL format, migration flag, and any Docker/Helm snippets). Otherwise, yes—SQLite will keep reminding you it’s not a fan of crowds.