https://prefect.io logo
Join Slack
Powered by
Hey folks, we've recently upgraded from `2.14.5` t...
# ask-community
m
Hey folks, we've recently upgraded from 
2.14.5
to 
2.16.0
and we have noticed a small issue with our task logs. Within the UI, task logs appear after a decently long delay. After looking into it we found that its the 
api/logs/filter
endpoint that is causing issues. We found that when filtering by 
task_run_id
it would take ~20 seconds, while without filtering it would take ~50ms. I've attached some examples of the bodies here:
Copy code
{"logs":{"level":{"ge_":0},"task_run_id":{"any_":["bb83c087-05bf-4ea5-9404-775c218d2e05"]}},"sort":"TIMESTAMP_ASC","offset":0,"limit":200}


{"logs":{"level":{"ge_":0}},"sort":"TIMESTAMP_ASC","offset":0,"limit":200}
n
hi @Mark Zheng - are you using prefect cloud or hosting server?
m
We are hosting our own server and using postgres
Just an update on this, I think we found the cause to potentially be the number of logs we have, although it is strange that it wasn't a gradual increase in delay but rather a sudden jump. We're going to look into setting an expiry time for log messages stored in the database.
As a side question, does prefect offer any support for this functionality?
n
you might be interested in implementing an additional loop service like this one, one that simply reads / cleans up old runs
did you see a drastic increase in the number of logs you were generating? or just latency
Open in Slack
PreviousNext
Powered by