Multiplayer Collaborative Canvas  ·  View 23 of 23  ·  Assurance

Failure Classes

Nine assumed failures, and the two the user has to be told about in words.

Editable source SVG draw.io All views
Detect Contain Recover The user sees Client drop Heartbeat miss Local op buffer Replay from seq "Reconnecting" Session host loss Lease not renewed Only its documents Re-host, epoch + 1 A 10 s pause Split authority Stale epoch write Rejected at the log Zombie self-detaches Nothing Log unavailable Append errors Fail closed on save Resume on recovery "Read-only" Control plane down Grant lookup fails 5 min cached grace New joins fail closed Cannot open new files Poison operation Apply crash loop Quarantine op + author Fix forward, replay One edit refused Fan-out degraded Publish lag Shed presence first Capacity added Cursors freeze Region loss Health probes Documents re-home RTO 15 min Up to 5 s lost Silent divergence Checksum mismatch Pin client version Re-snapshot client A brief reload Failure Classes — Detect, Contain, Recover, and What the User Sees Only two rows are visible as a degradation the user must be told about in words: read-only when the log cannot be written, and the loss window after a regional failover. v 1.0 · owner Platform Architecture · date 2026-10

Decisions

  • Fail closed on saving, fail open on viewing. Refusing an edit is honest; accepting one that may vanish is not (ADR-04).
  • A single client can never kill a document: a poison operation quarantines the operation and its author's connection, and the session stays alive.
  • Split authority is contained at the store rather than reconciled afterwards, which is why it is the one row in this table with 'the user sees: nothing' (ADR-03).

The two visible degradations

  • Read-only: the log cannot be written and the product says so within 5 s, rather than collecting edits it may lose.
  • Up to 5 s of operations lost after a regional failover, declared to users rather than discovered by them.

Risks

  • Silent divergence is the failure class with no natural alarm, which is why checksum reporting is a requirement and not an operational nicety (view 18).
  • A correlated reconnect is the failure that the recovery of another failure causes; it is the class most likely to be under-tested, and the reason admission control exists at all.
◀  Admission and Revocation All views
Solution Architecture v1.0 · Microsoft Azure · Platform Architecture · 2026-1023-failure-classes