<@ULVA73B9P> I am getting a lot of these issues: <...
# ask-marvin
c
@Marvin I am getting a lot of these issues: https://gist.github.com/valleybay/bb1bd9717596826e125b0343fb20c64a. Basically sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked happening every few minutes.
m
thought for 286.8 seconds
Thanks for sharing the logs — this is the classic SQLite “database is locked” pattern. SQLite uses file-level locks and under concurrent writes it will regularly produce this error. Prefect 3.x enables WAL mode and a busy timeout by default, which helps, but under any sustained concurrency you’ll still see these. A few targeted steps you can take right now: Short-term mitigations (if you must stay on SQLite) - Run a single API instance only - Ensure you’re not running multiple replicas or multiple uvicorn workers against the same SQLite file. - If you’ve got multiple “temporary servers” spinning up from different shells/machines, point everything at one shared, dedicated server instead (set
PREFECT_API_URL
for clients). - Put the DB on fast local disk - Avoid NFS/SMB/remote volumes; network FS exacerbates locking. - Increase SQLite’s lock wait/statement timeout - Prefect 3.x maps a database timeout to SQLite’s connect timeout; bump it to give locks more time to clear:
Copy code
export PREFECT_SERVER_DATABASE_CONNECTION_URL="sqlite+aiosqlite:////path/to/prefect.db"
  export PREFECT_SERVER_DATABASE_TIMEOUT=60    # try 60–120s
- Alternatively, include a timeout in the SQLite URL:
Copy code
export PREFECT_SERVER_DATABASE_CONNECTION_URL="sqlite+aiosqlite:////path/to/prefect.db?timeout=60.0"
- Reduce local contention - If you have background scripts or backup/indexers scanning the
.db
file, exclude it. - Investigate long transactions - Enable detailed logging to spot endpoints/queries that hold locks:
Copy code
export PREFECT_DEBUG_MODE=1
  export PREFECT_SERVER_DATABASE_ECHO=true
- Maintenance when the server is stopped - If the DB has grown large, a VACUUM/optimize can help:
Copy code
sqlite3 /path/to/prefect.db "PRAGMA optimize; VACUUM;"
Recommended long-term fix - Move to PostgreSQL for any non-trivial or multi-user setup. It eliminates SQLite’s locking limits and lets you scale safely. - Example DSN:
Copy code
export PREFECT_SERVER_DATABASE_CONNECTION_URL="<postgresql+asyncpg://USER:PASSWORD@HOST:5432/prefect>"
- With Docker Compose:
Copy code
services:
    postgres:
      image: postgres:15
      environment:
        POSTGRES_USER: prefect
        POSTGRES_PASSWORD: prefect_password
        POSTGRES_DB: prefect
      volumes:
        - postgres_data:/var/lib/postgresql/data

    prefect-server:
      image: prefecthq/prefect:3-latest
      environment:
        PREFECT_SERVER_DATABASE_CONNECTION_URL: "<postgresql+asyncpg://prefect:prefect_password@postgres:5432/prefect>"
      command: prefect server start
      ports:
        - "4200:4200"
      depends_on:
        - postgres
- If you use PgBouncer in transaction mode, set statement cache size to 0:
Copy code
export PREFECT_SERVER_DATABASE_SQLALCHEMY_CONNECT_ARGS_STATEMENT_CACHE_SIZE=0
Helpful references and similar reports - “Even basic examples result in excessive ‘database is locked’ logs” https://github.com/PrefectHQ/prefect/issues/18934 - “Long load times with Prefect Server and default SQLite” https://github.com/PrefectHQ/prefect/issues/16304 A few questions to tailor the advice: - Which Prefect version are you on? - Are you running a dedicated
prefect server start
instance, or relying on the “temporary server” that starts with local runs? - Is your database SQLite or Postgres today? If SQLite, where is the
.db
file stored (local disk vs. network volume)? - Any chance multiple API processes/replicas are pointed at the same SQLite file? If you can share those details, I can suggest precise settings and next steps for your setup.
c
This is a very simple setup. It's one docker container with a mounted volume (storage is locally on the hetzner server). It has nearly no load. We're talking max 1 job run per hour and this is still happening continually without any jobs even executing on Prefect.