Pierre L
11/07/2025, 9:41 AMMarvin
11/07/2025, 9:41 AMMarvin
11/07/2025, 9:51 AMpip install "prefect[redis]"
Docs: prefect-redis integration
2) Point Prefect to the Redis-backed broker and cache by setting these environment variables on your Prefect Server pods (API and services):
PREFECT_MESSAGING_BROKER=prefect_redis.messaging
PREFECT_MESSAGING_CACHE=prefect_redis.messaging
Settings reference: settings reference
3) Provide Redis/Dragonfly connection details used by prefect-redis. The integration exposes settings via RedisMessagingSettings and related helpers; you can typically provide a standard Redis URL or host/port/auth via env vars. See the integration docs above for the exact options accepted.
Notes for Helm/Kubernetes:
- If you previously deployed Bitnami Redis via a subchart, you can remove that and point Prefect to your Dragonfly service instead by setting the env vars above on the Prefect Server workloads. Ensure the Prefect server image includes prefect-redis.
- If you’re not running multiple replicas or don’t need cross-pod log/event distribution, you can skip Redis entirely and stay on the default in-memory broker.
A quick validation plan:
- Verify Dragonfly Streams end-to-end before switching Prefect:
- Check that XADD/XGROUP/XREADGROUP/XACK work with consumer groups.
- Stand up a test Prefect Server with:
PREFECT_MESSAGING_BROKER=prefect_redis.messaging
PREFECT_MESSAGING_CACHE=prefect_redis.messaging
- Run a few flows, watch that:
- Flow/task logs appear reliably in the UI
- Events render in the timeline
- With multiple server replicas, logs/events are visible across pods
- If you use concurrency limits or distributed locks, test those as well
Open questions for you (to tailor guidance):
- Which Prefect version and Helm chart are you on?
- Do you plan to run the Server in HA (multiple replicas)?
- Will you use Redis for just logs/events, or also for locks/concurrency?
If you share your version/chart and whether you’re aiming for HA, I can suggest a concrete values.yaml/env setup for the Prefect pods. Links for reference:
- Prefect Helm: Server on Helm and prefect-helm repo
- Prefect Redis integration: prefect-redisPierre L
11/27/2025, 1:31 PMPierre L
01/02/2026, 9:05 AMYou can't write against a read only replica.
I could not find explanations due to a misconfiguration, but cannot totally rule it out.