Hello everyone, is there any information on the data retention policy for prefect server? Thank you.

@Daniel Komisar what do you mean by data retention policy? What data specifically are you referring to? In general, Prefect operates on a hybrid execution model, which means that Prefect never receives your code or data, instead it operates on metadata. When you use Server, it means self-hosting so you decide on your own where and for how long you keep data such as logs. When talking about Prefect Cloud, we do have configurable log retention based on your Prefect Cloud plan. Does this answer your question?

Yes thank you.

A lot of Server users find that they need to maintain the database by clearing a log table and task run table a some regular basis (month, 2 months). There are no retention limits for Server

When you say “maintain the database”, what happens if this maintenance isn’t done? Is it just for our own benefit, or will queries slow down over time if we don’t?

Your API may fail because there’s no more memory to write new stuff in the database

Ah okay this sounds important, I have a couple questions. • How can well tell this is about to happen? • Any idea how much data it can hold? • Can you clear it out automatically for us?

We would have no idea on your infrastructure. It would be when the machine that holds the database runs out of memory. We don’t take responsibility for maintaining server deployments.

To give a little more context, we don’t have a very flow heavy workload. The only intensive part is that flows generate somewhat large logs. So I think this may not be a problem for us for a while, but it does sound like we will need to address it at some point and I’m not sure when that is.

Ah Cloud we have the 2 week or 3 week retention period. Enterprise gives 6 months though so stuff before that retention gets cleared. If you need persistence of logs more than that, some users ship the logs to DataDog

Thank you, understood.