Yes — with the right architecture, a self-hosted Prefect 3.x server can run in an HA-style setup and support background tasks. The short version: Recommended pattern - Multiple API server replicas behind a load balancer - Separate background service replicas from the API servers - PostgreSQL as the shared database - Redis for messaging/coordination - Task workers running separately for

.delay()

background tasks So your proposed setup — Redis + separate API server and services + multiple replicas — is aligned with how this should be done. Important caveat: the HA story depends on using Postgres and configuring Redis-backed coordination. You should not just scale

prefect server start

replicas with services enabled by default and hope it coordinates itself. --- What to scale 1. API servers - Safe to run multiple replicas - Run them without services:

prefect server start --host 0.0.0.0 --port 4200 --no-services

2. Server background services - Also can have multiple replicas - Run separately:

prefect server services start

- These rely on Docket coordination, which should point to Redis so only one instance performs a given scheduled service action at a time 3. Task workers - Your background tasks from

.delay()

are handled by task workers, not by the API containers themselves - You can run multiple task worker replicas and point them at the load-balanced API URL Given your existing background-task usage, this fits well: the HA server cluster stores task runs centrally, and multiple task workers can poll and execute them. --- Required infra/config Database - Use PostgreSQL for HA/self-hosted clustering - Run migrations once, not on every replica:

prefect server database upgrade -y

Then set:

PREFECT_SERVER_DATABASE_MIGRATE_ON_START=false

Redis You should configure Redis for: - event messaging - causal ordering - concurrency lease storage - Docket service coordination The especially important one for your question is:

PREFECT_SERVER_DOCKET_URL=<redis://redis:6379/1>

Without Redis-backed Docket, multiple

prefect server services start

replicas can duplicate scheduler/service work. --- High-level deployment pattern API deployment - 2+ replicas - behind LB / service - command:

prefect server start --no-services

Services deployment - 2+ replicas if desired - command:

prefect server services start

Task worker deployment - 1+ replicas depending on throughput - pointed to:

PREFECT_API_URL=<http://your-prefect-lb:4200/api>

--- Background tasks specifically Yes, background tasks still work in this topology. Flow is basically: 1. app/client submits

.delay()

2. API stores the task run in Postgres 3. task workers poll Prefect and pick up scheduled task runs 4. workers execute tasks independently of which API replica accepted the request So separating API from services does not break background tasks. --- Main caveats - Do not use SQLite for this - Do not run migrations on all replicas simultaneously - Do not scale service replicas without Redis-backed Docket configured - Watch Postgres connection counts as you add replicas - Redis cluster mode may not be supported in this path; standard Redis is the safer choice --- Useful docs/source The best reference here is Prefect’s self-hosting guidance: - Self-hosted Prefect 3 docs If you want, I can also sketch a Kubernetes layout for this specifically: -

prefect-api

deployment -

prefect-services

deployment -

task-worker

deployment - env vars / probes / service wiring That would probably map closely to how you're already running 3.4.14 in Kubernetes.

Thanks — that’s an important correction. You’re right to call this out: my earlier answer was too optimistic for background tasks specifically. Based on the current Prefect 3.x behavior and the issue you linked, the answer is more like: You can run an HA-style self-hosted server cluster, but background tasks are currently a problematic / not fully working area in that topology. Issue: - Issue #21218 What’s going on There are really two separate things here: 1. General server HA / separation of API and services - Multiple API replicas - Separate

prefect server services start

- Shared Postgres - Redis-backed coordination 2. Background tasks - The

.delay()

/ task-worker path - This is where the issue you linked becomes relevant The current implementation has some sharp edges around how server-side background/service coordination works in multi-replica self-hosted deployments. In particular, separating API and services does not currently guarantee background-task behavior will work correctly just because Redis is present. Important nuance -

prefect server start --no-services

means the API pods are not running the normal server services - Those services must run elsewhere - Coordination depends on Docket - Docket defaults to

memory://

, which is not suitable for multi-process / multi-replica setups - Even with Redis configured for Docket, issue

#21218

indicates background-task behavior in this architecture is still not reliable / not working as expected So if your question is: “Can we run multiple replicas of API + multiple replicas of services + Redis + background tasks, in a supported working HA setup?” The practical answer today is: Not reliably for background tasks, based on the current bug/limitation. What I would say today - HA for the API/services plane: yes, that architecture exists - HA + self-hosted + background tasks: currently there is a known issue / limitation - If background tasks are required, I would not treat this topology as fully supported/stable yet Safer options right now 1. Single server/services process for the control plane, plus multiple task workers - less HA - simpler - more likely to behave correctly 2. Avoid relying on background tasks for this architecture and instead use more traditional Prefect flow/worker execution patterns where possible 3. If you do experiment with HA anyway: - use Postgres - use Redis for Docket:

PREFECT_SERVER_DOCKET_URL=redis://...

- separate API from services - but understand this does not resolve the known issue you linked Bottom line Because of #21218, I would currently answer no to “is this working for background tasks in HA self-hosted mode?” even though parts of the server architecture are designed for horizontal scaling. If you want, I can help you with one of these next: - a recommended non-HA background task deployment pattern for your current Kubernetes setup - a best-effort HA topology with all the caveats called out - or a workaround architecture to preserve async fan-out without using Prefect background tasks directly