<@ULVA73B9P> how long does prefect store logs, and...
# ask-marvin
p
@Marvin how long does prefect store logs, and in general past job entries?
m
thought for 82.2 seconds
Based on the research and available information, here's what you need to know about Prefect's log and job retention: For self-hosted Prefect servers: 1. Default Behavior: - There is no built-in automatic cleanup mechanism - All logs and flow run data are stored indefinitely in your database by default - The database size will continue to grow unless managed 2. Database Management: - You're responsible for managing the database size and implementing retention policies - This can be done through database-level maintenance scripts - If using PostgreSQL, you can implement cleanup using tools like
pg_cron
3. Best Practices for Self-Hosted Setups: - Implement your own retention policy using database maintenance scripts - Be careful when deleting data due to foreign key relationships - Focus on cleaning up: - Old flow run logs - Completed flow runs beyond your needed retention period - Task run data that's no longer relevant 4. Configuration Options: - The server's event retention period can be configured via
server.events.retention_period
(defaults to "P7D" - 7 days) - This primarily affects real-time events, not historical logs If you need to implement a cleanup strategy, you'll need to create database maintenance scripts that carefully handle the relationships between tables. When deleting data, it's recommended to: 1. Start with logs (safest to delete) 2. Then handle flow runs and related data 3. Ensure your cleanup respects foreign key constraints For a robust solution, consider implementing a rolling retention policy based on your specific needs (e.g., keeping the last 30 days of logs, or the last 90 days of flow runs).