Anatomy

Diagrams of boxes and arrows describe an idea. These build it. Each piece is the real thing, running: a transformer you can type into, laid out in 3D from its own numbers; a web stack you can overload. The 3D dissections walk through a landmark paper or system component by component, trace every figure to its source, and end with experiments of their own.

15 models 7 AI & ML5 Architecture3 Mechanistic Interpretability 1,222 parts and components 353 links into the libraries

All15 AI & ML7 Architecture5 Mechanistic Interpretability3

Any Language models 3 Neural network foundations 2 Training & alignment 2 Performance & capacity 1 Web & API systems 1 Distributed systems 3 Causal methods 2 Hallucination & faithfulness 1

Any Working model 10 Simulation 5

Runs the real algorithm end to end on your own input.

3 models for “pre-registered experiments”.

Working model · 3D AI & ML · Training & alignment

Anatomy of Neural Scaling Laws

Kaplan et al. said spend on size; Hoffmann et al. said spend on both. 108 trained miniatures, three estimators and one question: what does the answer depend on?

Eleven character-level Transformers and 562 training runs laid out as a landscape of loss; three estimators find the compute-optimal size on it. A pre-registered desk tests what moves the exponent: t…

32 components · Intermediate 23 Oct 2026

Working model · 3D Mechanistic Interpretability · Hallucination & faithfulness

Why 722 becomes 719

A real 0.5B language model opened up at the moment it copies a number from its context: the heads that carry the digit, the votes that choose it, the sampler that sometimes doesn't, and a guard that checks every digit as it is written.

Two people paste the same research notes into a small model and get 722 and 719. A 3D dissection of Qwen2.5-0.5B at the moment it copies the number, with pre-registered experiments on five models: wh…

24 components · Intermediate 18 Oct 2026

Working model · 3D AI & ML · Language models

Anatomy of the Transformer

Attention Is All You Need, dissected: the whole encoder-decoder in 3D, every matrix a real one from a trained model running in your browser, every number from the paper traced to its sentence.

The 2017 Transformer, built in 3D from its own numbers: a trained miniature of the paper's exact encoder-decoder translates your English number into German while the camera walks through 29 steps, fr…

172 components · Intermediate 15 Oct 2026