ASHIK
08/22/2025, 11:14 AMMarvin
08/22/2025, 11:14 AMMarvin
08/22/2025, 11:22 AMPREFECT_LOGGING_TO_API_ENABLED=false
- Or persist it to your profile:
prefect config set PREFECT_LOGGING_TO_API_ENABLED=false
- Reduce log volume
- Raise the logging level (default is INFO):
PREFECT_LOGGING_LEVEL=WARNING
# or persist:
prefect config set PREFECT_LOGGING_LEVEL=WARNING
- Ensure results aren’t persisted
- Prefect 3 defaults to not persisting results, but if you’ve changed it:
PREFECT_RESULTS_PERSIST_BY_DEFAULT=false
prefect config set PREFECT_RESULTS_PERSIST_BY_DEFAULT=false
- Also avoid setting persist_result=True
on tasks/flows unless you truly need it.
- Avoid artifacts unless necessary
- Artifacts (e.g., markdown/json artifacts) are stored in the DB and can be large. Don’t create them unless you need them.
Events retention (server-side)
- Events can grow quickly; set a shorter retention to keep the events table small:
PREFECT_SERVER_EVENTS_RETENTION_PERIOD="3 days"
prefect config set PREFECT_SERVER_EVENTS_RETENTION_PERIOD="3 days"
- Docs: Settings reference
What you cannot disable
- Flow runs, task runs, and their states cannot be turned off; they’re required for scheduling, retries, and the UI. The best approach is retention/pruning.
Prune existing data (OSS Server)
- There isn’t a built-in CLI “prune” today; if you need to reclaim space immediately, you can delete old rows and VACUUM at the database level. Start with logs and events (test on a backup first!):
Example PostgreSQL maintenance:
-- Delete logs older than 14 days
DELETE FROM log WHERE timestamp < NOW() - INTERVAL '14 days';
-- Delete events older than 7 days (match your configured retention)
DELETE FROM event WHERE occurred < NOW() - INTERVAL '7 days';
-- Reclaim space
VACUUM (VERBOSE, ANALYZE) log;
VACUUM (VERBOSE, ANALYZE) event;
- If you have very old runs you no longer need, you can delete completed flow runs (which cascades to task runs, states, and logs) via the API. If you want, I can share a small Python snippet that filters for old completed runs and deletes them in batches.
A couple of clarifying questions so I can tailor this:
- Are you on Prefect Cloud or self-hosted Prefect Server?
- Which Prefect version (2.x vs 3.x)? I assumed 3.x above.
- Do you know which tables are largest (e.g., log
, event
, artifact
, flow_run
, task_run
)? If you can share a quick breakdown, I can suggest a targeted plan.
Helpful docs
- Settings reference: Prefect 3 Settings
- Database maintenance (overview): DB maintenance (guidance on pruning and VACUUM)
If you confirm your setup, I’ll provide exact commands or a ready-to-run cleanup script.