Training & Alignment
21 min
Muon and MuonClip: The Optimizer That Broke Adam's Monopoly on LLM Pretraining
For eight years Adam owned every serious pretraining run. Then a competitive-speedrun optimizer that orthogonalizes its own gradient updates scaled from a 124M-parameter toy to a trillion-parameter model with zero loss spikes. He…