Module 13 / Replication

Module 13 · lesson 4 of 9 · Read 10 min

Electing a leader

Who takes over, and how to avoid two leaders.

In Module 5, Tiffin's primary database died at 8:15pm and a replica took over. We looked at it from the outside: how long it took, what customers saw, what got lost. This lesson opens up the mechanism. Something has to decide that the leader is dead, pick a successor, and make sure everyone uses it.

The hard part isn't choosing a new leader. It's making sure there is only ever one. A database with two leaders, each accepting writes the other never sees, is worse than a database with none. That situation is called split brain, and most of this lesson is about avoiding it.

Three jobs

Every failover, automatic or manual, does the same three things:

Detectis the leader really dead?
→
Choosewhich follower takes over?
→
Switchpoint everyone at it
The three jobs of a failover. Each one can go wrong on its own.