Hey <@ULVA73B9P>, is this error log a known issue?...
# ask-marvin
s
Hey @Marvin, is this error log a known issue?
Error running periodically process followers task
It's followed by
Periodically process followers task cancelled successfully
And after this, we see no more task_run_recorder logs or tasks created in UI/DB. Wondering if it hit an issue and silently died?
m
thought for 164.9 seconds
Short answer: those messages come from the Task Run Recorder’s periodic “followers” processor. It runs every few seconds to recover “follower” task-run events whose “leader” event hasn’t arrived yet. - “Error running periodically process followers task” = the periodic loop hit an exception (it’s caught and logged; the loop normally keeps retrying). - “Periodically process followers task cancelled successfully” = the service is shutting down, so it cancels that background task during cleanup. On their own, these logs are not fatal. However, if you then stop seeing any
task_run_recorder
logs and no new tasks appear in the UI/DB, that usually means the Task Run Recorder consumer (or the whole server process) exited or got stuck—often due to an underlying DB/broker/connectivity issue. The cancellation message is a symptom of the shutdown, not the cause. Context on the feature: it was added as a periodic task to handle “lost followers” here: PR #15100. A few next checks to narrow this down: - Grab the full traceback right after “Error running periodically process followers task” — that will usually point to the root cause (e.g., DB connection, lock, or query error). - Look for any “service starting/restarting” messages after the cancellation log; if the process restarted, you should see the periodic task “starting” log again. - Confirm your DB is healthy (connections available, no locks/timeouts) around that timestamp. - If you can, bump logging to DEBUG for
prefect.server.services.task_run_recorder
to get more detail, then reproduce. - What Prefect version are you on? There have been fixes in this area; upgrading to the latest 3.x patch is recommended. Could you share: - Your Prefect version and whether this is Prefect Cloud or OSS Server - The full traceback around that error - Any adjacent DB/broker logs around the same time With that, I can help pinpoint whether this is a transient DB/broker hiccup, a service crash, or something we should escalate.