Yu Shao Pang
09/03/2025, 8:20 AMMarvin
09/03/2025, 8:20 AMMarvin
09/03/2025, 8:29 AMprefect config view --show-defaults --show-sources
Common knobs (set as env vars or in your profile) to relieve pool pressure:
# Connection URL (Postgres strongly recommended for production)
PREFECT_SERVER_DATABASE_CONNECTION_URL=<postgresql+asyncpg://USER:PASS@HOST:5432/DBNAME>
# SQLAlchemy pool sizing
PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_SIZE=20
PREFECT_SERVER_DATABASE_SQLALCHEMY_MAX_OVERFLOW=20
PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_TIMEOUT=30
PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_RECYCLE=3600
# Connection dialing timeouts (seconds)
PREFECT_SERVER_DATABASE_CONNECTION_TIMEOUT=60
PREFECT_SERVER_DATABASE_TIMEOUT=60
Notes:
- The exact names above match the config tree keys you’ll see in prefect config view
. If you’re unsure, set the env var then run prefect config view
again to confirm it’s picked up.
- After changing settings, restart your Prefect server.
2) Reduce load and free connections temporarily
- Pause or stop workers so the API isn’t servicing scheduling/assignment load while you delete.
- If you’re under heavy load, temporarily disable a few background services to free DB connections, then re‑enable afterward. For example:
PREFECT_SERVER_SERVICES_SCHEDULER_ENABLED=false
PREFECT_SERVER_SERVICES_TRIGGERS_ENABLED=false
PREFECT_SERVER_SERVICES_FLOW_RUN_NOTIFICATIONS_ENABLED=false
Restart the server, perform the delete, then restore to true.
3) Delete dependent resources first
Large/active pools can take longer to delete and may hold locks. Try:
- Ensure no deployments are actively targeting that work pool (reassign or delete them).
- Delete work queues in the pool first:
- List queues: prefect work-queue ls
(identify the ones for that pool)
- Delete each by ID: prefect work-queue delete <id>
- Then delete the pool:
prefect work-pool delete "<POOL_NAME>"
4) If Postgres, check DB health during the operation
- Ensure you’re not hitting max_connections; increase DB connections or reduce Prefect pool size to fit within DB limits.
- Check for long‑running or idle-in-transaction sessions that might be holding locks or ties up connections (pg_stat_activity).
- If you use a connection proxy (e.g., pgbouncer in transaction mode), confirm it’s not limiting effective concurrency.
If you’re on SQLite
- SQLite will frequently lock under write pressure and is not recommended for deleting larger/active metadata sets. If that’s your setup, consider migrating to Postgres before retrying the delete.
If you need a quick workaround
- If the delete keeps failing, you can pause the pool to keep it inert until we tune settings.
- As a last resort, we can step through a careful DB‑level cleanup, but I’d prefer to avoid raw SQL unless necessary — happy to help if it comes to that.
References (Prefect 2.x docs)
- Host Prefect (2.x)
- Work Pools (2.x)
- Settings (2.x)
Finally, if you can share:Marvin
09/03/2025, 8:29 AMprefect config view --show-defaults --show-sources
for the server.database
section
- A short snippet of the API server traceback for the 500
- DB type/version and any RDS/pg metrics around the time
…I can give you concrete values for pool sizing/timeouts that fit your environment and help confirm next steps.
Optional note: If upgrading is on the table, Prefect 3.x includes a number of server performance and connection management improvements. I can outline a safe upgrade path from 2.x once we get you unblocked.