AI & ML ↗ Architecture ↗

Anatomy

Diagrams of boxes and arrows describe an idea. These build it. Each piece is the real thing, running: a transformer you can type into, laid out in 3D from its own numbers; a web stack you can overload. The 3D dissections walk through a landmark paper or system component by component, trace every figure to its source, and end with experiments of their own.

14 models 6 AI & ML5 Architecture3 Mechanistic Interpretability 1,190 parts and components 335 links into the libraries
All14 AI & ML6 Architecture5 Mechanistic Interpretability3
Category Any Language models 3 Neural network foundations 2 Training & alignment 1 Performance & capacity 1 Web & API systems 1 Distributed systems 3 Causal methods 2 Hallucination & faithfulness 1
Kind Any Working model 9 Simulation 5

4 models for “MLP”.

Locating a fact: a still of the machine Working model · 3D Mechanistic Interpretability · Causal methods

Locating a fact

Causal tracing and rank-one editing on a real 0.5B language model: where restoring a state brings a fact back, where editing a matrix changes it, and whether those are the same place.

A 3D dissection of causal tracing and ROME on Qwen2.5-0.5B. On 72 pre-selected facts the early MLP site at the last subject token reproduces, much earlier in depth than in GPT-2 XL; editing works bes…
31 components · Intermediate 21 Oct 2026
Where the answer forms: a still of the machine Working model · 3D Mechanistic Interpretability · Causal methods

Where the answer forms

A real 0.5B language model read mid-computation by two lenses, the model's own last step and one trained per layer, and then changed by patching to see where the answer is actually made.

Two lenses decode a model's middle layers into words. A 3D dissection of Qwen2.5-0.5B where it writes one token: the tuned lens is far closer to the model than the logit lens, but on 90 pre-registere…
160 components · Intermediate 19 Oct 2026
Why 722 becomes 719: a still of the machine Working model · 3D Mechanistic Interpretability · Hallucination & faithfulness

Why 722 becomes 719

A real 0.5B language model opened up at the moment it copies a number from its context: the heads that carry the digit, the votes that choose it, the sampler that sometimes doesn't, and a guard that checks every digit as it is written.

Two people paste the same research notes into a small model and get 722 and 719. A 3D dissection of Qwen2.5-0.5B at the moment it copies the number, with pre-registered experiments on five models: wh…
24 components · Intermediate 18 Oct 2026