A real GPT-2, scaled down to 0.94 million parameters, running in your browser. Type anything and watch it think.
This is not an animation of GPT-2; it is GPT-2. The tokenizer is byte-level BPE with GPT-2's own pre-tokenisation pattern, the network is GPT-2's block (pre-LayerNorm, causal multi-head attention, a 4x GELU MLP, residual adds, a tied read-out), and the forward pass runs in your browser on whatever you type. Only the sizes changed: 4 blocks instead of 12, 128 wide instead of 768, 1,024 tokens instead of 50,257, trained on children's stories so that a model this small can still write. Six panels follow one token through tokenisation, embedding, attention, the MLP, the logit lens and sampling, each with the theory and links into the library.
A type press. The newest slug on the rail is the token being computed; on each of four stepped tiers (the four blocks) threads tighten from earlier slugs to it as hard as that block attends to them, a dial reads how much the block's MLP rewrites it, and five plungers in a gantry rise to the five most likely next tokens. The drawn cap flies onto the rail and the press runs again.
What is real. Every number is computed in the page from the trained weights: a 940,800-parameter GPT-2 (int8, one scale per row) whose JavaScript forward pass is checked against PyTorch's logits to within 1e-5 before the page is built. Attention threads, dial readings, plunger heights, the logit lens and the sampling draw all come from that pass.
