Hi,
Looking for recommendations on HA (High Availability) and FT (Fault Tolerance) setup for Prefect Open-Source self-hosting :-
1. How can we ensure high availability of Prefect locally running servers? Can we set up Prefect cluster?
• If yes, can there be a quick transition of workflows to another running instance if one of the instances goes down?
• If no, then how do we ensure that queued and running instances are handled gracefully and resumed?
• How to gracefully restart a failed Prefect instance?
2. What is the fault tolerance mechanism for Prefect self-hosted installation? If an instance fails, does it persist and pick up queued or running instances at the time of failure for re-execution on its own?
Any pointers will help.