https://prefect.io logo
#prefect-server
Title
# prefect-server
d

Dotan Asselmann

02/16/2022, 8:20 AM
Hey is there a standard way (maybe by configuration) to manage the data retention in the postgres db? Our pvc is almost full and I realised nothing ever deleted
a

Anna Geller

02/16/2022, 8:57 AM
A copy from https://stackoverflow.com/questions/70680055/cleaning-prefect-pg-data-when-using-prefect/70680394#70680394 there are a couple of ways you can clean up the Postgres database: • you can manually delete old records, especially logs from the flow run table using DELETE FROM in SQL, • you can do the same in an automated fashion, e.g. some users have an actual flow that runs on schedule and purges old data from the database, • alternatively, you can use the open-source pg_cron job scheduler for Postgres to schedule such DB administration tasks (@Dotan Asselmann - this seems to answer your question), • you can also do the same using GraphQL: you would need to query for flow run IDs of "old" flow runs using the 
flow_run
 query, and then execute 
delete_flow_run
 mutation, • lastly, to be more proactive, you can reduce the number of logs you generate by generally logging less (only logging what's needed) and setting the log level to a lower category, e.g. instead of using DEBUG logs on your agent, switching to INFO should significantly reduce the amount of space consumed by logs in the database.
d

Dotan Asselmann

02/16/2022, 7:23 PM
thanks
2 Views