Ariel Farbman
11/10/2025, 10:24 AMMarvin
11/10/2025, 10:24 AMMarvin
11/10/2025, 10:32 AM_alembic_tmp_work_queue is created with a UNIQUE constraint on name. If your existing data contains duplicate work queue names (often across different work pools), the copy into that temp table fails with the UNIQUE constraint error.
Why this happens
- On SQLite, Alembic can’t just alter constraints; it recreates tables and copies data. If there are multiple rows with the same work_queue.name, the copy into the temp table fails on the unique index.
- This typically shows up when resetting or upgrading a local SQLite-backed server that has multiple work queues with the same name.
A couple of paths forward — pick based on whether you need to preserve data:
Option A — You can wipe the DB (fastest)
If you’re okay losing all server data (flows, runs, queues, etc.), delete the SQLite DB file or the Docker volume, then restart the server so it creates a clean schema.
1) Stop the Prefect server (and anything else connecting to the DB).
2) Identify the database path or connection URL:
- Run:
prefect config view --show-sources --show-defaults
Look for PREFECT_SERVER_DATABASE_CONNECTION_URL (alias PREFECT_API_DATABASE_CONNECTION_URL). If it starts with sqlite+aiosqlite:///, the file path follows that prefix.
3) Delete the file (or the Docker volume if you mounted one). Examples:
- Direct file removal:
rm /path/to/your/prefect.sqlite
- If you used Docker Compose and a named volume for Prefect’s home (common), remove the volume:
docker compose down
docker volume ls | grep prefect
docker volume rm <your_prefect_volume_name>
4) Start the server again (or run a reset):
prefect server database reset -y
or just start the server and it will initialize the schema.
Docs:
- Prefect Server CLI
- Database maintenance
- Settings reference
Option B — You need to keep your data
If you want to preserve what’s in SQLite, you’ll need to remove or rename any duplicate work queue names before running the migration/reset again.
1) Find the SQLite file path as above, then open it:
sqlite3 /path/to/your/prefect.sqlite
2) Check for duplicates by name:
SELECT name, COUNT(*) as cnt
FROM work_queue
GROUP BY name
HAVING cnt > 1;
Optionally, check how they distribute across pools:
SELECT name, work_pool_id, COUNT(*) as cnt
FROM work_queue
GROUP BY name, work_pool_id
HAVING cnt > 1;
3) For any duplicated names, either delete or rename rows until all name values are unique:
- Example rename approach (be careful; adjust to your data):
-- See the rows first
SELECT id, name FROM work_queue WHERE name = 'duplicate_name';
-- Rename specific ones
UPDATE work_queue
SET name = name || '_' || id
WHERE id IN (<ids to rename>);
4) Exit sqlite3, then try:
prefect server database reset -y
If you prefer to only run migrations:
prefect server database upgrade
Option C — Use Postgres for the server
If you’re running a longer-lived self-hosted server, we recommend Postgres instead of SQLite. Set PREFECT_SERVER_DATABASE_CONNECTION_URL to a Postgres DSN, then run:
prefect server database reset -y
to initialize. This avoids SQLite’s ALTER TABLE limitations and is more robust overall.
A few checks to prevent this in the future
- Make sure you’re running the CLI in the same environment/profile the server uses (so you’re resetting the intended DB). You can verify with:
prefect config view --show-sourcesMarvin
11/10/2025, 10:32 AMprefect config view --show-sources, what is the value for PREFECT_SERVER_DATABASE_CONNECTION_URL?
Once I know those, I can give you exact commands (including any Docker volume names/paths) to fix this quickly.Ariel Farbman
11/10/2025, 11:53 AMMarvin
11/10/2025, 11:53 AMMarvin
11/10/2025, 11:54 AMprefect server database reset -y is running against SQLite (hence the sqlite3 IntegrityError), but your actual server DB is Postgres. The server CLI operates directly on the database connection configured in the CLI’s environment, not via the API URL. Because your CLI environment isn’t pointing at Postgres, it’s trying to reset a default SQLite DB instead and hits the Alembic/SQLite uniqueness issue.
Since you’re on Prefect 3.2.12, using Postgres, and you’re OK wiping data, here are two clean ways to reset:
Option 1 — Drop the Postgres data volume (clean slate)
- Stop Prefect server and Postgres containers
- Remove the Postgres volume, then start fresh
Example:
docker stop <prefect_server_container> <postgres_container>
# Remove the Postgres data volume (replace with your actual volume name)
docker volume ls | grep postgres
docker volume rm <your_postgres_volume_name>
# Restart Postgres and Prefect server
docker start <postgres_container>
docker start <prefect_server_container>
When the Prefect server starts against an empty Postgres DB, it will initialize the schema.
Option 2 — Run a targeted DB reset against Postgres
- Stop the Prefect server to avoid active connections
- Run the reset pointing explicitly at your Postgres DSN
Using a one-off Prefect CLI container (recommended to ensure correct deps):
docker stop <prefect_server_container>
docker run --rm \
--network <your_docker_network> \
-e PREFECT_SERVER_DATABASE_CONNECTION_URL="postgresql+asyncpg://<user>:<pass>@<postgres_host>:5432/<db_name>" \
prefecthq/prefect:3.2.12 \
prefect server database reset -y
Or from your host (only if your local env has asyncpg installed and can reach Postgres):
export PREFECT_SERVER_DATABASE_CONNECTION_URL="postgresql+asyncpg://<user>:<pass>@<postgres_host>:5432/<db_name>"
prefect server database reset -y
Important notes
- The API URL you shared (PREFECT_API_URL='<http://prefect-server:4200/api>') only configures the HTTP client; it does not tell the server database CLI where your DB is. For DB operations, the CLI uses PREFECT_SERVER_DATABASE_CONNECTION_URL (alias: PREFECT_API_DATABASE_CONNECTION_URL).
- To verify what the CLI will target, run this in the exact environment where you run `prefect server database reset`:
prefect config view --show-sources --show-defaults | grep DATABASE_CONNECTION_URL
If it’s empty or shows a sqlite+aiosqlite:// URL, that explains the SQLite error.
Docs for reference:
- Server CLI
- Database maintenance
- Settings reference (see PREFECT_SERVER_DATABASE_CONNECTION_URL)
If you want, share your Postgres container name, network, and DSN (redact secrets), and I’ll provide an exact docker run command for Option 2.Ariel Farbman
11/10/2025, 11:58 AMMarvin
11/10/2025, 11:59 AMMarvin
11/10/2025, 12:03 PMcurl -X POST \
"<http(s)://<your-prefect-server>/api/admin/clear-database>" \
-H "Content-Type: application/json" \
-d '{"confirm": true}'
- Fully reset the schema (drop and recreate all tables):
# Drop all Prefect tables
curl -X POST \
"<http(s)://<your-prefect-server>/api/admin/drop-database>" \
-H "Content-Type: application/json" \
-d '{"confirm": true}'
# Recreate the schema
curl -X POST \
"<http(s)://<your-prefect-server>/api/admin/create-database>" \
-H "Content-Type: application/json" \
-d '{"confirm": true}'
Docs:
- /admin/clear-database
- /admin/drop-database
- /admin/create-database
Notes:
- These are POST requests and require {"confirm": true} in the JSON body.
- If you’ve put the API behind auth, include your auth header.
- Stop your workers/agents before running this to avoid new data being written mid-operation.
Option B — Use the CLI against your RDS URL
This drops and recreates all Prefect tables (not the DB itself).
- Stop the Prefect server (to release connections).
- Run the reset with your Postgres DSN:
# From a Prefect container (ensures asyncpg is available)
docker run --rm \
--network <your_network_if_needed> \
-e PREFECT_SERVER_DATABASE_CONNECTION_URL="postgresql+asyncpg://<user>:<pass>@<rds-host>:5432/<db_name>" \
prefecthq/prefect:3.2.12 \
prefect server database reset -y
Docs:
- Server CLI
- Database maintenance
What gets deleted
- Flows, deployments, flow runs, task runs, logs, artifacts, blocks, UI state — everything managed by Prefect in that DB.
Recommended procedure
1) Take an RDS snapshot (just in case).
2) Stop Prefect server and any workers/agents.
3) Run either the admin endpoints or the CLI reset.
4) Start Prefect server and workers again.
If you share your Prefect API URL and whether your server requires auth, I can give you exact curl commands tailored to your setup.Ariel Farbman
11/10/2025, 12:14 PMMarvin
11/10/2025, 12:14 PMMarvin
11/10/2025, 12:15 PM