I am working on setting up a HA Prefect server in Kubernetes and I am currently running 3 API Server pods, 1 Background Services pod, 1 Redis pod, and a Postgres server in a separate VM outside of Kubernetes.
The flow runs are functioning fine, and I can see the flow run state accurately in the UI as well as the logs being populated in real time. However, task runs are stuck in a 'Pending' state in the UI for what seems like exactly 15 mins after the flow run completes (even though they actually ran fine). I traced a specific task run through the redis logs and I can see it go from Pending > Running > Completed and all 4 Prefect server pods acknowledging each state change (see attached photo). The actual task run only takes about 40 seconds.
I then checked the task run in Postgres and I can see that it was not updated to the 'Completed' state until 15 minutes after the flow completed (see query below), so it seems that this is what the UI is reading from (for task state at least) in my current configuration. Is there some setting that I missed configuring for my HA configuration so that task runs are updated in the UI more frequently or is there something else going on? For comparison, in the UI the flow run kicked off at 180039 and completed at 180113.
Been beating my head against this for a few days so any insight is much appreciated, thanks!
SELECT created, updated, state_type, state_name
FROM task_run
WHERE id = '0198cdca-6483-772c-8a04-c985f57c2e24';
created | updated | state_type | state_name
-------------------------------+-------------------------------+------------+------------
2025-08-21 180050.571844+00 | 2025-08-21 181551.127393+00 | COMPLETED | Completed
(1 row)