Working model · 3D
Mechanistic Interpretability · Hallucination & faithfulness
Why 722 becomes 719
A real 0.5B language model opened up at the moment it copies a number from its context: the heads that carry the digit, the votes that choose it, the sampler that sometimes doesn't, and a guard that checks every digit as it is written.Two people paste the same research notes into a small model and get 722 and 719. A 3D dissection of Qwen2.5-0.5B at the moment it copies the number, with pre-registered experiments on five models: wh…