Time cannot go backwards, and other lies production believed

Two postmortems (Cloudflare 2017, Azure leap day 2012), three Linux kernel commits, the Go and Rust standard-library design records, Kubernetes leader election, and the clock architectures of Google, Meta, AWS and CockroachDB, reconstructed into a reference clock stack, four decision records with flip conditions, a three-class failure catalogue and a build ladder. After reading, an architect can audit a codebase for wall-clock subtraction, decide when a lease needs a fencing token, and price commit-wait against uncertainty restarts on their own hardware.

What you get out of it

• The Go monotonic-clock gap was on file for 15 months (issue #12914, Oct 2015) and became an accepted design 25 days after Cloudflare's leap-second outage; the proposal rejects two-API designs because clock resets make misuse fail on every machine simultaneously, defeating redundancy.
• Rust walked the same road in reverse: it forced monotonicity in std in 2019 because platform clocks were untrustworthy, then removed the enforcement in 2022 when its worst-case cost measured above 100x, choosing saturating arithmetic as the survivable poison.
• The layer that implements time is not exempt from time bugs: the 2012 leap second deadlocked the Linux timekeeping path and fired CLOCK_REALTIME timers a second early, and the mainline commit message preserves the industry's mid-incident workaround, date -s "`date`".
• Kubernetes leader election ships with its clock caveat in the package comment: no fencing guarantee; exclusivity that matters must be enforced by the resource via a monotonically increasing token, not by the leaseholder's own good behaviour.
• Bounded uncertainty is purchasable: Spanner's measured epsilon averaged 4 ms on GPS-and-atomic hardware, making a ~5 ms commit wait affordable, while CockroachDB on 500 ms commodity bounds restarts transactions instead; cloud microsecond sync now moves that flip condition.

Scope

Why this, now. Cloud providers now rent microsecond-class clock error bounds (AWS Time Sync plus ClockBound), which reopens the wait-versus-retry ordering decision most teams last made on 500 ms NTP assumptions.

What it does not cover. NTP and PTP protocol engineering and security, GPS spoofing and resilience, timezone and tzdata handling, and regulatory clock-sync mandates in finance.