I'm running a Prefect server and when we run some tasks generating a lot of logs (like 600 lines in a second), the UI stops to show them at some moment and there are no logs for the following tasks : they are not delayed, they just never appear (the prefect cli doesn't show as well).
Is it a known issue ? Has someone run into the same kind of problem here ?
Thank you !
01/21/2022, 1:29 PM
It’s the first time I see this issue. Your Server probably doesn’t manage to write all those logs at once to the database. Can you check the Server logs to see whether you see some indication of this?
As a hacky workaround, perhaps you could add some time.sleep() in between log messages to give Server a bit more time to process… And of course, you could always set a different log level for this flow to limit the amount of logs that need to be written in the first place, if your use case allows it.
01/21/2022, 1:49 PM
and thank you for your quick response !
I will definitely check the server logs.
including time.sleep() is not an option but for the moment we have decreased the log level.