Kaplan et al. (2020) and Hoffmann et al. (2022) fitted power laws to language-model losses and drew opposite advice about how to split a fixed compute budget between model size and data. This page rebuilds the disagreement at miniature scale: eleven decoder-only Transformers from 1.5 thousand to 590 thousand parameters, trained on the site's own prose under two training protocols, three seeds each, drawn as a landscape of loss with the fitted frontiers laid over it. Your own sentence is scored live by every model, the three estimators of Hoffmann et al. are refitted in the page, and a research desk pre-registers three questions the papers left open (what counts as a parameter, whether the three approaches agree, and whether the schedule matters) and reports each with its seeds, its bootstrap intervals and its verdict.
The machine
Height is how far a run's loss is below the worst loss in the study, so the best size for any budget is the crest along its diagonal. The measured landscape (model size across, training tokens into the distance) sits beside the loss the fitted law predicts and what is left over; five tubes lie over it: the compute-optimal frontier by Hoffmann et al.'s three approaches and the slopes of Kaplan et al. and of equal scaling. A ledger shows the eleven models' parameter counts, embedding share and the compute the 6N rule omits; one 13.8k-parameter model is laid open with its real weights running on your text.
What is real. The eleven models are PyTorch-trained float16 weights run forward in plain JavaScript and checked against PyTorch (logits, attention, per-character loss). The estimators (power-law and parabola fits, Approaches 1, 2 and 3, Kaplan's L(N, D)) are the same code that produced the research results and are checked against numpy and scipy, against a noiseless surface with a known optimum, and against every derived number in the text. Literature numbers are sourced to verbatim quotes from the arXiv LaTeX; the hypotheses and verdict rules were written before the grid ran.
Working model · 3D
AI & ML · Training & alignment
