Prefect does not require Redis by default. If you do use Redis with Prefect 3.x, it expects standalone Redis — not Redis Cluster. Short answer: - Standalone Redis: yes - Redis Cluster: no, not currently supported - Redis Sentinel: not something I found as an officially supported cluster mode here either; the key point is Prefect’s Redis integrations are built around non-cluster Redis URLs The Prefect code explicitly detects cluster-style Redis URLs and raises a

NotImplementedError

, so cluster support is intentionally disabled right now. What Redis is used for in Prefect Redis is optional and mainly used for distributed/server-side features like: - messaging/event transport - event ordering - concurrency lease storage - worker cleanup queue storage - some distributed locking/storage patterns If you’re just running a basic/single-instance setup, you usually do not need Redis. If you want Redis-backed behavior, use a normal Redis instance, for example:

bash
<redis://host:6379/0>

or TLS:

bash
<rediss://host:6379/0>

If helpful, I can also show you: 1. which Prefect features need Redis, or 2. the exact env vars to configure standalone Redis for Prefect server

If you mean “I want to run multiple Prefect server instances — is Redis necessary?” then: Yes — practically, you should use Redis for a multi-server Prefect setup. Why: - Prefect can run without Redis using in-memory components - but those in-memory backends are local to each server instance - with multiple Prefect server replicas, they won’t share state correctly for things like: - messaging - event handling - ordering - concurrency lease coordination - worker cleanup queues So for multi-instance Prefect server, use: - Postgres for the database - standalone Redis for shared coordination/messaging - not Redis Cluster So the recommendation is:

text
multiple Prefect server instances
        + shared Postgres
        + shared standalone Redis

Important: Redis Cluster is currently not supported by Prefect’s Redis integrations. If you want, I can give you a minimal HA/self-hosted architecture for Prefect server with multiple replicas.

If you already set

PREFECT_API_DATABASE_CONNECTION_URL

, that only configures the database. For a multi-server Prefect setup, yes, you should also configure the Redis-backed settings. Otherwise Prefect will keep using in-memory defaults for messaging/coordination, which are not shared across replicas. So the short answer is: -

PREFECT_API_DATABASE_CONNECTION_URL

→ not enough by itself for multi-server - You should also set Redis-related Prefect settings - You do not need Redis Cluster; use standalone Redis What to configure for multiple Prefect server replicas At minimum, set:

bash
PREFECT_MESSAGING_BROKER=prefect_redis.messaging
PREFECT_MESSAGING_CACHE=prefect_redis.messaging
PREFECT_SERVER_EVENTS_CAUSAL_ORDERING=prefect_redis.ordering
PREFECT_SERVER_CONCURRENCY_LEASE_STORAGE=prefect_redis.lease_storage

And point them to Redis with either:

bash
PREFECT_REDIS_MESSAGING_URL=<redis://redis:6379/0>

or separate fields like:

bash
PREFECT_REDIS_MESSAGING_HOST=redis
PREFECT_REDIS_MESSAGING_PORT=6379
PREFECT_REDIS_MESSAGING_DB=0

If you also run multiple background service processes, then set:

bash
PREFECT_SERVER_DOCKET_URL=<redis://redis:6379/1>

That one matters for coordinating background services. If you leave it as the default

memory://

and run multiple service processes, you can get duplicate scheduling / duplicate automation behavior. A practical example

bash
PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>

PREFECT_MESSAGING_BROKER=prefect_redis.messaging
PREFECT_MESSAGING_CACHE=prefect_redis.messaging
PREFECT_SERVER_EVENTS_CAUSAL_ORDERING=prefect_redis.ordering
PREFECT_SERVER_CONCURRENCY_LEASE_STORAGE=prefect_redis.lease_storage

PREFECT_REDIS_MESSAGING_URL=<redis://redis:6379/0>

PREFECT_SERVER_DOCKET_URL=<redis://redis:6379/1>

Rule of thumb - One Prefect server instance: DB URL may be enough - Multiple Prefect server instances: add Redis-backed messaging/settings - Multiple background service instances: also set

PREFECT_SERVER_DOCKET_URL

to Redis If you want, I can turn this into a docker-compose example for HA Prefect server.

Yes — those are two different settings for two different subsystems. Your understanding is correct: -

PREFECT_REDIS_MESSAGING_URL

→ Redis used by Prefect’s messaging-related components -

PREFECT_SERVER_DOCKET_URL

→ Redis used by background service coordination Using different Redis DB numbers like

/0

and

/1

is a good practice to keep those concerns separated. So this is a sensible setup:

bash
PREFECT_REDIS_MESSAGING_URL=<redis://redis:6379/0>
PREFECT_SERVER_DOCKET_URL=<redis://redis:6379/1>

A few notes: - They can point to the same Redis server - Using different logical DBs is recommended for separation, but not because Prefect strictly requires it - This is for standalone Redis, not Redis Cluster One small correction: in actual env vars, don’t use Slack link formatting like

<redis://...|redis://...>

. Just use the raw value:

bash
PREFECT_REDIS_MESSAGING_URL=<redis://redis:6379/0>
PREFECT_SERVER_DOCKET_URL=<redis://redis:6379/1>

If helpful, I can also give you a complete recommended env var set for running multiple Prefect API + service replicas behind Docker/Kubernetes.

Here’s a recommended baseline for a self-hosted, multi-replica Prefect Server setup. Assumptions - Prefect 3.x - multiple Prefect server/API replicas - shared Postgres - shared standalone Redis - possibly multiple background service replicas - Docker/Kubernetes style deployment Recommended env vars

bash
# -----------------------------
# Database
# -----------------------------
PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://prefect:prefect@postgres:5432/prefect>

# Important in multi-replica setups:
# only one process should run migrations on startup, or disable and run migrations separately
PREFECT_API_DATABASE_MIGRATE_ON_START=false


# -----------------------------
# Redis-backed messaging/event coordination
# -----------------------------
PREFECT_MESSAGING_BROKER=prefect_redis.messaging
PREFECT_MESSAGING_CACHE=prefect_redis.messaging
PREFECT_SERVER_EVENTS_CAUSAL_ORDERING=prefect_redis.ordering
PREFECT_SERVER_CONCURRENCY_LEASE_STORAGE=prefect_redis.lease_storage

# Shared Redis for messaging/event/concurrency
PREFECT_REDIS_MESSAGING_URL=<redis://redis:6379/0>


# -----------------------------
# Background service coordination
# -----------------------------
# Required if running multiple background service processes/replicas
PREFECT_SERVER_DOCKET_URL=<redis://redis:6379/1>


# -----------------------------
# API / server behavior
# -----------------------------
PREFECT_API_URL=<http://prefect-server:4200/api>

# Optional: if serving UI/API behind an ingress or reverse proxy, set this to the external URL instead
# PREFECT_UI_API_URL=<https://your-prefect.example.com/api>


# -----------------------------
# Logging / telemetry (optional)
# -----------------------------
PREFECT_LOGGING_LEVEL=INFO


# -----------------------------
# Profiles / explicit server mode (optional)
# -----------------------------
PREFECT_SERVER_API_HOST=0.0.0.0
PREFECT_SERVER_API_PORT=4200

If your Redis requires auth/TLS Use URLs like:

bash
PREFECT_REDIS_MESSAGING_URL=<redis://username:password@redis:6379/0>
PREFECT_SERVER_DOCKET_URL=<redis://username:password@redis:6379/1>

or TLS:

bash
PREFECT_REDIS_MESSAGING_URL=<rediss://username:password@redis:6379/0>
PREFECT_SERVER_DOCKET_URL=<rediss://username:password@redis:6379/1>

Recommended deployment pattern 1. API/server replicas All API replicas can share the same env vars above. 2. Background services If you run separate service processes, they should use the same DB/Redis config, especially:

bash
PREFECT_SERVER_DOCKET_URL=<redis://redis:6379/1>

3. Migrations In HA setups, avoid every replica trying to migrate on startup. Common pattern: - one migration job/init container runs migrations - all normal replicas use:

bash
PREFECT_API_DATABASE_MIGRATE_ON_START=false

Minimal architecture

text
[Prefect API replica 1] \
[Prefect API replica 2]  ---> shared Postgres
[Prefect API replica 3] /

[all replicas] ----------> shared Redis DB 0 for messaging/concurrency
[service replicas] ------> shared Redis DB 1 for docket/service coordination

Practical notes - Use Postgres, not SQLite, for multi-replica server - Use standalone Redis, not Redis Cluster - Splitting Redis DBs (

/0

and

/1

) is a good organizational choice - Make sure

prefect-redis

is available in the image/environment If you’re using the official Prefect image, that integration is generally available. Useful references - Prefect docs: Prefect documentation - Prefect server settings reference: Settings reference - Prefect repository: Prefect GitHub If you want, I can next give you either: - a docker-compose example, or - a Kubernetes Deployment/Helm-style example for this setup.