Anatomy

Diagrams of boxes and arrows describe an idea. These build it. Each piece is the real thing, running: a transformer you can type into, laid out in 3D from its own numbers; a web stack you can overload. The 3D dissections walk through a landmark paper or system component by component, trace every figure to its source, and end with experiments of their own.

13 models 6 AI & ML5 Architecture2 Mechanistic Interpretability 1,159 parts and components 306 links into the libraries

All13 AI & ML6 Architecture5 Mechanistic Interpretability2

Any Language models 3 Neural network foundations 2 Training & alignment 1 Performance & capacity 1 Web & API systems 1 Distributed systems 3 Causal methods 1 Hallucination & faithfulness 1

Any Working model 8 Simulation 5

3 models for “attention heads”.

Working model · 3D AI & ML · Language models

Anatomy of Induction Heads

In-context Learning and Induction Heads, dissected: a two-layer attention-only Transformer in 3D, every matrix a real one from a model trained in the repository, with seven pre-registered hypotheses tested on 14 training runs.

A two-layer attention-only Transformer, built in 3D from its own numbers, completes [A][B] … [A] → [B] on your text while the camera walks the previous-token head, the K-composition and the induction…

56 components · Advanced 20 Oct 2026

Working model · 3D Mechanistic Interpretability · Hallucination & faithfulness

Why 722 becomes 719

A real 0.5B language model opened up at the moment it copies a number from its context: the heads that carry the digit, the votes that choose it, the sampler that sometimes doesn't, and a guard that checks every digit as it is written.

Two people paste the same research notes into a small model and get 722 and 719. A 3D dissection of Qwen2.5-0.5B at the moment it copies the number, with pre-registered experiments on five models: wh…

24 components · Intermediate 18 Oct 2026

Working model · 3D AI & ML · Language models

Anatomy of the Transformer

Attention Is All You Need, dissected: the whole encoder-decoder in 3D, every matrix a real one from a trained model running in your browser, every number from the paper traced to its sentence.

The 2017 Transformer, built in 3D from its own numbers: a trained miniature of the paper's exact encoder-decoder translates your English number into German while the camera walks through 29 steps, fr…

172 components · Intermediate 15 Oct 2026