Hey there, for the run history, how long it keeps the logs? If a flow runs extensively long time, will it continue to finish even though the run history is wiped out? How can i track the logs for those long running flows?

Do you mean the logs for all run? A flow run should continue to finish even though the run history is wiped out. You can track the logs either though your CLI or the UI

Thanks, @Khuyen Tran. The UI has been wiped out for this flow, I dont see anywhere else to find them in the UI. What is the CLI command to show the logs for a specific flow run?

So you have wiped the history of all your flow runs?

i am under the impression that there is a limit of how long the run history keeps the previous runs( unfinished runs), my flow has been running for > 15 days, so today when I went back to the UI and check, the run history is wiped out. Not me proactively cleaned the run history.

I checked with the team and the logs for flow runs that are more than 7 days old are deleted for free users. So you will not be able to achieve the logs for runs > 15 days

Thanks, but is the flow still running? how can i check the status of the flow if it is more than 7 days?

Hello Ray, we noticed that you are on the Aven tenant and that has a log retention period of 14 days. We're confirmed with our team what the expected behaviour is here.

That is correct. Thanks for looking at it. The flow is a hyperparameter search with bayesian optimization. And it takes more than 15 days to finish all the search rounds. I have in total 4 runs that are started at similar time and they are all disappeared now. From our EC2 console, i can still see 2 clusters for the 4 runs that are still running.

Hello Ray, I've confirmed with the team what is occurring here. The standard plan currently offers a 14 day retention period for flow run data and logs. The countdown for the retention period begins at the

scheduled_start_time

of the flow run. Due to the fact that this flow run takes longer than 14 days to complete, the logs and data are cleared prior to its completion. We recommend reaching out to your account executive or sales@prefect.io to request increasing the retention period, which should resolve this issue on a forward basis.

So it is not only about the logs, it is also about the flow that is currently running, even though the clusters are with Aven? So it means we cannot perform persistent ML flows with prefect which exceeds the retention period?

Correct, the flow runs and logs will be cleared after the retention period.

Got it. Thanks. we will pay attention to the retention period next time.