@Marvin What does the Prefect use internally for scheduling. How does it achieve scheduler HA?

Prefect uses a sophisticated scheduling mechanism that can handle various types of schedules, including those defined by iCal recurrence rules (RRules). These rules allow for complex scheduling patterns, such as repeating tasks on specific days or intervals. Internally, Prefect utilizes the

dateutil

library to manage these recurrence rules, which provides flexibility in defining schedules with complex logic like the last weekday of each month or every other day of the week. Read more about scheduling in Prefect 3.x. Regarding high availability (HA) for scheduling, Prefect's approach involves prioritization within work queues to ensure that critical tasks are submitted and executed before less critical ones. This is achieved through priority settings for different queues, allowing Prefect to manage and execute tasks based on defined priority levels. While this helps in managing task execution order, it doesn't inherently provide HA for the scheduler itself. An issue was noted in Prefect's GitHub repository regarding the behavior when the scheduler becomes unavailable. Specifically, if the scheduler is unavailable for 10 seconds, scheduled flows terminate, which is not ideal for high availability. The expected behavior would be for the flow to wait for the scheduler to return and resume at the earliest opportunity. See the related GitHub issue. In summary, Prefect uses RRules for scheduling and manages task execution priorities to handle workloads effectively. However, achieving true scheduler HA might require additional configurations or considerations beyond the built-in prioritization and scheduling logic.

Bad bot, thanks for crap answer 🙂

hi @Pratap - the scheduler is one of many server-side services. when you create a flow run from your deployment, the scheduler will handle queuing up the run at the right time for whatever worker should execute it, so is your question how to make the server highly available?

Can the scheduler run on multiple servers? If not, What if that one server goes down? Can you run multiple servers with scheduler roles so that it has HA so that one server going down won't impact it. In kubernetes world, this is fine because orchestration takes care of spinning up the scheduler pod in another node. But, k8s just abstracting and hiding of the fact that scheduler is single point of failure if it can scheduler can only run on the

you can run the scheduler independently of the server, i’d check out the helm chart in prefect-helm, there’s a recent issue where someone asked a similar question

Unfortunately, we don't use kubernetes, so can't use helm. Does it support only 1 pod/server? How do we achieve HA with it.

can you explain what you mean when you say HA? do you just mean running more than one server at a time?