Training & Alignment
26 min
The Bias-Variance Tradeoff Is a Special Case: Double Descent, Benign Overfitting, and Grokking
A network that fits ImageNet with randomly shuffled labels should not generalise on real ones. It does. That single experiment invalidated the textbook account of why machine learning works, and the three phenomena that replaced …