Triple-machine paxos on high availability architecture - Quorum

As one of the most popular distributed storage systems, Hadoop Distributed File System (HDFS) has structure to accommodate mass data processing. However, the master/slave architecture only has one single Namenode for metadata management, which has put HDFS in the risk of Single Point of Failure (SPoF). To resolve the problem, there have been a few solutions based on multiple Namenodes. Nevertheless, all of them have some shortcomings in terms of failover efficiency and metadata consistent among Namenodes. Hence, we present a new metadata management schema with two symmetrical Namenodes, which reduces recovery time loss and increases failover efficiency by providing symmetrical read/write service and consistent metadata. To resolve metadata consistency problem, we introduce a third-party node—quorum node, which works together with the other two Namenodes to compose our high availability (HA) architecture Quorum, design a modified triple-machine Paxos algorithm, and finally apply the algorithm to our architecture. Our tests show that the proposed architecture and triple-machine Paxos are feasible.