Short answer (Prefect 3.x): these Prefect API/server services read from the Redis events stream: - ReactiveTriggers service — evaluates event-driven automations - Source: triggers service - EventPersister service — persists events to the database for history/audit - Source: event persister - EventLogger service (disabled by default) — logs events for debugging - Source: event logger - Distributor — pushes live events to WebSocket subscribers (used by the UI); subscribes to the event stream in-process - Source: event stream distributor Notes and clarifications: - The Actions service does not read the events stream; it consumes the separate actions stream produced by automations: - actions service - Logs are on a separate logs stream handled by a LogDistributor: - log distributor - Consumer group names for stream readers are defined centrally: - consumer names OSS vs Cloud: the same services consume the event stream in both; Cloud just runs/manages them for you. If you’re troubleshooting Redis directly, you should see consumer groups for the reactive triggers and event persister, and ephemeral subscriptions from the distributor on webserver processes.

Likely cause: the persister isn’t “reading in 10k chunks.” In Prefect 3.x the Redis consumer reads exactly 1 event per XREADGROUP call, then the persister batches for DB writes. With a 5s flush interval, you’ll typically see only the handful of events accumulated in that window (often ~100–200), even if your batch size is 10,000 and the backlog is large. Key details (code-verified): - Redis consumer read size is hard-coded to 1 - XREADGROUP uses count=1: https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-redis/prefect_redis/messaging.py#L408-L414 - Pending messages are also processed one-at-a-time: https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-redis/prefect_redis/messaging.py#L398-L402 - Persister’s batch_size controls DB write flushing, not Redis read size - Handler setup: https://github.com/PrefectHQ/prefect/blob/main/src/prefect/server/events/services/event_persister.py#L114-L120 - Handler queues each message and flushes when batch_size reached or timer expires: https://github.com/PrefectHQ/prefect/blob/main/src/prefect/server/events/services/event_persister.py#L225-L241 - Flush timer (defaults to ~5s): https://github.com/PrefectHQ/prefect/blob/main/src/prefect/server/events/services/event_persister.py#L208-L215 - DB writes are further split into “safe batches” by DB parameter limits (e.g., SQLite ~60 events; Postgres ~1–2k per insert), but this is usually not what causes the ~100–200 number: - https://github.com/PrefectHQ/prefect/blob/main/src/prefect/server/events/storage/database.py#L306-L329 Other factors that can slow consumption: - If there are many pending (unacked) messages (e.g., after a crash), the consumer processes them one-by-one before reading new ones: https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-redis/prefect_redis/messaging.py#L396-L404 - The consumer uses a block timeout per read; not usually the limiter with a large backlog, but present: https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-redis/prefect_redis/messaging.py#L411-L414 What you can do: - Scale out event consumption: run additional API/server replicas so multiple EventPersister consumers share the load via the Redis consumer group. Throughput increases roughly linearly with the number of replicas. - Ensure you’re on Postgres (not SQLite) for event storage; SQLite’s parameter limits keep per-insert batches very small. - Check for a large “pending” backlog in the persister’s consumer group (and stale consumers) and clean them up if needed: - In Redis: XINFO GROUPS <events-stream>, XPENDING <events-stream> <group>, XINFO CONSUMERS <events-stream> <group> - If you need a faster catch-up and you’re comfortable customizing, consider patching prefect-redis to increase the XREADGROUP count and XAUTOCLAIM batch; it’s currently hard-coded to 1 “for now” in the code above. If you can share: - DB backend (Postgres vs SQLite) - Number of API/server replicas - Whether you see a large XPENDING for the persister’s consumer group …I can suggest more targeted steps.