😃 RAFT - It's About Reliable Master-Slave 😃

# [ $davids.sh ] · message #269

• @ [ $davids.sh ] · # 1691

  In short, the database node set selects a leader, all write operations go through it, but it doesn't confirm the write until a majority agrees. In case of the leader's death, one of the followers is selected as the next leader. In case of network failure, if a majority is in one part of the network, that part of the network will continue to work.

  Such a system can be trusted and will self-recover in case of failure.

  In this CAP situation, we chose Consistency and a bit of Partition Tolerance.

  If we choose a node for the entire database as the leader, there will be no multi-master. However, for master-slave replication, this would be the most reliable scheme. But in modern conditions, it would also require an asynchronous replication mechanism to reduce the load for read-only databases.

  If we choose a node for a table as the leader, everything will become even worse because data consistency will simply slow down the entire system.

  "So what should we do?" - there are other distributed system protocols that inherently assume multi-master (Couchbase, Cassandra, ElectricSQL, etc.). But for a RAFT-based system to work in a multi-master setup, we need to add a bit more magic.

  For example, as implemented in CockroachDB, YDB, and the new ScyllaDB, we can split our table into smaller groups (e.g., ranges by IDs), and now each individual node is responsible only for its group. This means that many nodes, each taking a small part, are now responsible for a single entire table.

  And there you have it – multi-master.

• @ Ivan ITK 🚫 · # 1700

  • Raft (extended), 2014: https://www.usenix.org/system/files/conference/atc14/atc14-paper-ongaro.pdf
  • Zookeeper, 2010: https://www.usenix.org/legacy/event/usenix10/tech/full_papers/Hunt.pdf
  • Spanner, 2012: http://nil.lcs.mit.edu/6.824/2020/papers/spanner.pdf
  • CockroachDB, 2020: https://dl.acm.org/doi/pdf/10.1145/3318464.3386134
  • COPS, 2011: http://nil.lcs.mit.edu/6.824/2020/papers/cops.pdf
  • Impossibility of Distributed Consensus with One Faulty Process, 1985: https://dl.acm.org/doi/pdf/10.1145/3149.214121
  • Time, clocks, and the ordering of events in a distributed system, 1978: https://dl.acm.org/doi/pdf/10.1145/359545.359563

• @ Ivan ITK 🚫 · # 1701

  A collection of materials from CockroachDB engineers, posted at the very beginning of their journey, when the database was about a year old.

• @ Ivan ITK 🚫 · # 1702

  This is all on the topic of rangers, rafts, and solving database scaling problems.