Christian Dalsvaag
11/27/2025, 7:33 AMMarvin
11/27/2025, 7:33 AMMarvin
11/27/2025, 7:38 AMPREFECT_API_URL for clients).
- Put the DB on fast local disk
- Avoid NFS/SMB/remote volumes; network FS exacerbates locking.
- Increase SQLite’s lock wait/statement timeout
- Prefect 3.x maps a database timeout to SQLite’s connect timeout; bump it to give locks more time to clear:
export PREFECT_SERVER_DATABASE_CONNECTION_URL="sqlite+aiosqlite:////path/to/prefect.db"
export PREFECT_SERVER_DATABASE_TIMEOUT=60 # try 60–120s
- Alternatively, include a timeout in the SQLite URL:
export PREFECT_SERVER_DATABASE_CONNECTION_URL="sqlite+aiosqlite:////path/to/prefect.db?timeout=60.0"
- Reduce local contention
- If you have background scripts or backup/indexers scanning the .db file, exclude it.
- Investigate long transactions
- Enable detailed logging to spot endpoints/queries that hold locks:
export PREFECT_DEBUG_MODE=1
export PREFECT_SERVER_DATABASE_ECHO=true
- Maintenance when the server is stopped
- If the DB has grown large, a VACUUM/optimize can help:
sqlite3 /path/to/prefect.db "PRAGMA optimize; VACUUM;"
Recommended long-term fix
- Move to PostgreSQL for any non-trivial or multi-user setup. It eliminates SQLite’s locking limits and lets you scale safely.
- Example DSN:
export PREFECT_SERVER_DATABASE_CONNECTION_URL="<postgresql+asyncpg://USER:PASSWORD@HOST:5432/prefect>"
- With Docker Compose:
services:
postgres:
image: postgres:15
environment:
POSTGRES_USER: prefect
POSTGRES_PASSWORD: prefect_password
POSTGRES_DB: prefect
volumes:
- postgres_data:/var/lib/postgresql/data
prefect-server:
image: prefecthq/prefect:3-latest
environment:
PREFECT_SERVER_DATABASE_CONNECTION_URL: "<postgresql+asyncpg://prefect:prefect_password@postgres:5432/prefect>"
command: prefect server start
ports:
- "4200:4200"
depends_on:
- postgres
- If you use PgBouncer in transaction mode, set statement cache size to 0:
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_STATEMENT_CACHE_SIZE=0
Helpful references and similar reports
- “Even basic examples result in excessive ‘database is locked’ logs” https://github.com/PrefectHQ/prefect/issues/18934
- “Long load times with Prefect Server and default SQLite” https://github.com/PrefectHQ/prefect/issues/16304
A few questions to tailor the advice:
- Which Prefect version are you on?
- Are you running a dedicated prefect server start instance, or relying on the “temporary server” that starts with local runs?
- Is your database SQLite or Postgres today? If SQLite, where is the .db file stored (local disk vs. network volume)?
- Any chance multiple API processes/replicas are pointed at the same SQLite file?
If you can share those details, I can suggest precise settings and next steps for your setup.Christian Dalsvaag
11/27/2025, 7:47 AM