Hi all! I'm looking into a single tool to unify da...
# prefect-community
k
Hi all! I'm looking into a single tool to unify data flows and Prefect looks interesting! From the installation docs I can see it's simply a
pip install
. I am curious what high-availability options there are in case the master/scheduling/coordinator(?) node fails, and where the configurations/flows are stored that one can restore from.
k
Hello @kelv! Welcome to the Prefect Community. Most Prefect users will utilize a Kubernetes Agent on a cluster to implement a high-availability setup. There’s a great tutorial on scaling out with this particular agent you can read in the documentation, as well as more information on that particular Agent. Additionally, Prefect provides a variety of storage options for your flows including local storage, Azure, AWS, GCS, and Docker. I hope these resources get you going in the right direction!
👍 1