A follower in a healthy 5-node cluster is partitioned for 90 seconds and spends that time timing out and incrementing its term. The partition heals. What happens in the next few milliseconds?

Second by second

The isolated follower cannot reach anyone, so it campaigns, fails, and campaigns again. With a 150 to 300 ms election timeout, 90 seconds of isolation leaves it carrying a term perhaps 300 higher than the rest of the cluster - and in Raft, term number dominates everything.

On reconnection it broadcasts a vote request at that inflated term. Every member, leader included, follows the rule "if you see a higher term, step down and adopt it". So:

1. The healthy leader converts to follower, through no fault of its own.
2. The returning node asks for votes - and loses, because its log is behind and voters reject a candidate whose log is less up to date.
3. The cluster now has no leader, at a term nobody wanted, and waits out a randomised election timeout before electing one - usually the previous leader.

What the user sees

A write-unavailability gap of roughly one election timeout plus one replication round, so tens to hundreds of milliseconds, and in-flight writes that return "leader changed" and must be retried. One node, which was never needed and never wins, forced the cluster to pause. Repeated across a flapping link, it looks like random leadership churn with no correlated cause - the classic symptom of this bug.

What stops it

Pre-vote. Before incrementing its term, a candidate runs a trial election: would you vote for me if I asked? Voters answer from the same rules they would really use, including the log-up-to-date check, and the candidate only increments its term and campaigns for real if it could win. A node returning with a stale log gets no pre-votes, so it never raises its term, and the healthy leader never hears a higher term. The mechanism is described in the Raft dissertation (2014) and is on by default in etcd.

The companion control, needed for the opposite direction, is check-quorum: a leader that has not heard from a majority within an election timeout steps down on its own, so a leader isolated on the minority side stops claiming leadership instead of serving stale leases.

What would have to be true for it to self-heal

Nothing - it does self-heal, in milliseconds, which is exactly why it survives in production for years. Each occurrence is cheap, the repair is automatic, and no single event is worth an investigation. Only the aggregate hurts: leadership changes invalidate read leases, reset client connections and abort in-flight batches. The signal to graph is leader-change rate per hour, with an alert on a step change rather than on any single election.

When this is the wrong fix

If the member is campaigning because its disk is slow enough to miss heartbeats, pre-vote removes the symptom and keeps the sick member in the quorum, where it continues to set your commit latency. Pre-vote protects the cluster from a healthy-but-isolated node; it hides a degraded one. Keep the per-member commit-latency alert.