Attention Is All You Need, dissected: the whole encoder-decoder in 3D, every matrix a real one from a trained model running in your browser, every number from the paper traced to its sentence.
A complete, granular dissection of Vaswani et al. (2017). The paper's architecture, unchanged except for its sizes, was trained to translate English number words into German, a task that forces reordering ('forty two' becomes 'zwei und vierzig'). Every tensor of its forward pass on your input is drawn on a 3D stage as a block of cells, with the encoder, decoder, shared embedding and tied output layer where the paper's Figure 1 puts them. A 29-step article walks the camera through tokens, embeddings, positional encoding, queries, keys and values, scaled dot products, softmax, multi-head attention, residual connections and LayerNorm, the feed-forward network, masking, encoder-decoder attention, the tied output layer, beam search, label smoothing, the warm-up schedule, dropout, complexity, results and the parameter count, each with the paper's equations and every figure from the paper hover-linked to the sentence it comes from. A research desk then tests three open questions with new, pre-registered experiments.
Two towers on a table, the paper's Figure 1 made of numbers. The encoder (left) and decoder (right) each have a spine, the residual stream, with sub-layers built out to the side: weight matrices sit under the activations they produce with columns aligned, attention heads stand in pods of scores, softmax bars and outputs, orange arcs carry the encoder's memory into each decoder layer, and a teal line joins the shared embedding at the bottom to its transpose at the top. Cell colour and height are the values.
What is real. Everything on the stage is computed in the page from trained weights: a 61,472-parameter encoder-decoder with the paper's architecture and recipe (post-LN, sinusoidal positions, three-way tied embeddings, Adam 0.9/0.98/1e-9 with the warm-up formula, dropout 0.1, label smoothing 0.1, beam 4 with alpha 0.6), trained in PyTorch for 4,000 steps on a CPU. Its JavaScript forward pass, greedy and beam search match PyTorch to within 1e-5 on reference inputs (test.mjs, 66 checks), and the same tests recompute every derived number in the text: parameter counts, the learning-rate peak, the positional-encoding rotation identity and the variance of q·k. Paper figures are quoted from the arXiv v7 LaTeX source and listed in the page's fact ledger. The research-desk experiments are reproducible from research/.
