November 3, 2010

Quorum algorithm of the Zookeeper

The Apache Zookeeper is a coordination service for distributed applications, like a Google's Chubby. Many projects uses zookeeper, and we (Apache Hama) also uses zookeeper for barrier synchronization of Bulk Synchronous Parallel computing framework.

Today, I surveyed more about paxos and dynamic quorum of Zookeeper project, to renaming the class name of org.apache.hama.zookeeper.QuorumPeer. Because of documentation is not enough, I didn't know what is the meaning of quorum and the term of "quorum" was somewhat odd to me.
But, the "org.apache.hama.zookeeper.QuorumPeer" is proper name!! xD

So, what is the Quorum and why do we need a Quorum?

According to Wikipedia, Quorum is the minimum number of members of a deliberative body necessary to conduct the business of that group. Ordinarily, this is a majority of the people expected to be there, although many bodies may have a lower or higher quorum.

As you know, a Fault-Tolerant mechanism is one of the important functions of distributed system. Quorum algorithm is used to prevent the split-brain condition. When split-brain condition occurrs, according to the Quorum algorithm, zookeeper determines the "Primary Patrition" and "Secondary Partition". Then, the servers in primary group are receives and processes user's request, and the servers in secondary group are becomes read-only mode. When recovered from split-brain condition? They will be merged to one partition again. Interanally, zookeeper uses atomic broadcast protocol instead of Paxos.

1 comment:

  1. Anonymous8/4/17 12:34

    Hi Edward,

    Thanks for the above post. I am currently working on trying to use zookeeper in a two node cluster. I have my own cluster formation algorithm running on the nodes based on configuration. We only need Zookeeper's distributed DB functionality.

    1. Is it possible to use Zookeeper in a two node cluster ? Do you know of any solutions where this has been done ?
    2. Can we still retain the zookeepers DB functionality without forming a quorum ?

    Note: Fault tolerence is not o main concern in this project. If one of the nodes go down we have enough code logic to run without the zookeeper service.

    Would greatly appreciate any help.