how can we control the size of the logs to make sure they do not get to large on one hand, but to make sure it keeps enough history ?

Hey @YD, are you talking about the server log table and you’re asking because it takes too much memory? A lot of users just truncate entries past 1 month or 2 weeks.

how do they do this? is it something I specify in the config.toml ?

By going to the database and deleting the tables. Like managing the database.

Manually? in the "Interactive API" tab ?

No no as in running queries on the PostgreSQL database that the API hits.

@YD I'm running my system on Kubernetes so I set up a cronjob deployment to connect to the postgres database and clean up certain tables older than 90 days. The SQL for that ends up being approximately

DELETE FROM log WHERE timestamp < NOW() - INTERVAL '90 days';

I did not setup any DB to keep the log, just using the default Prefect server logging (on my VM) how do I access it?

Hey @YD, when you start the server, the server is backed by a postgres database. There will be a container for that and the logs live in that database. So you can run SQL queries on that database. What Sam is suggesting is to delete the entries using SQL

The prefect postgres is running in a Docker, right ? Do I set the cronjob on the server hosting the Prefect and the docker? do you maybe know the script I should add to crontab? including the docker command (

docker exec -it

...) .

I wouldn’t have an example for that as I haven’t used cron myself, but I imagine it is finding a way to programmatically access that postgres db and running the query. Maybe use psql to run the command? Let’s see if @Sam Cookhas more to add.

It looks like I might be able to run using

docker exec -it temp_data_processing_postgres_1  sh -c "psql -U username -d mydatabase -c 'SELECT * FROM log limit 10;' "

but I think I need the

username

and the

mydatabase

prefect is using for the log do you know what is it ?

You need run the cronjob somewhere with psql access that can connect to the postgres server, usually easiest from the postgres server itself or from your baremetal machine. Your command looks good, the default user and password should be

prefect

and `test-password`and the database is

prefect_server

https://github.com/PrefectHQ/prefect/blob/master/src/prefect/config.toml You can also set environment variables for the postgres parameters if you don't want to type them on command line

this command works

docker exec -it temp_data_processing_postgres_1  sh -c "psql PGPASSWORD=test-passwordand -U prefect -d prefect_server -c 'SELECT * FROM log limit 10;' "

thanks