Hey everyone! Can anyone tell me if the Orion API and/or Scheduler and/or UI support running multiple replicas? Is anyone running them in an HA config already?

Great question! One engineer from our team thinks about this problem a lot and we may address this in the second half of the year, but due to the complexity of doing it well it's not on an immediate roadmap in general, the OSS Prefect 2.0 orchestration layer will be easier to scale than e.g. Prefect 1.0 due to simpler architecture - you should be able to deploy multiple API servers talking to the same Postgres backend and load balance the incoming requests We definitely welcome your input if you are willing to contribute and share your solution with the community

You should be able to deploy multiple API servers talking to the same Postgres backend and load balance the incoming requests

To clarify, is this saying you can do that right now and it’s just the scheduler that needs work, or are you describing the target state for later development?

I don't want to give you a wrong answer, but generally speaking, a scheduler service is part of the Orion API server and you could in theory (to achieve HA) have multiple uvicorn servers and load balance the requests what I meant with complexity is e.g. ensuring how those multiple replicas work together e.g. I know that we use idempotency keys to avoid duplicate work (the same run being picked up twice) but I don't know whether this would be enough to prevent duplicate work when running multiple Orion servers This is definitely a hard problem (as anything distributed/HA) and my main point is that I don't want to give you a sense that you can just deploy multiple replicas, add a load balancer on top and you're done, there is a lurking complexity here to ensure it all works well together

Okay, from the sounds of things you’ve at least baked in a concept of idempotency. I do appreciate the caution, definitely better than overconfidence in this area. I might just give it a shot in dev for a bit and see what happens. I was under the impression I could run the scheduler and the API separately though. I saw the

--no-scheduler

and

--scheduler

flags and assumed I could run Orion as the API only and then one with the scheduler only.

Indeed, it only shows that you already know more about it than I do 👏 it seems you can run a separate Orion server without UI, scheduler, mark late runs and analytics services

Anything I make that’s shareable I definitely will. Once I’ve got my current setup working nicely and proven in a prod like environment then I’m planning to wrap most of it in a Kubernetes operator and open source it. Don’t want to do that until I know it actually works though as I don’t want to push out rubbish and have some poor unsuspecting dev pick it up.

All of the services can be run separately from the API to scale them individually

Good info, thanks. I guess I can pool them with pgbouncer to ease that. I’m more comfortable having that problem than I am having something I can’t allow to scale.

You can call

python -m "prefect.orion.services.scheduler"

Oh cool! Thanks!

And similarly for any of the other services. I don’t think logging will be set up correctly yet though as we’ve not polished this.