A complete, granular dissection of He, Zhang, Ren and Sun (2016), Deep Residual Learning for Image Recognition. The paper's CIFAR-10 network with n = 3 (ResNet-20), unchanged except for halved widths, was trained with the paper's recipe on the UCI handwritten digits. Every feature map, kernel and batch-norm statistic of its forward pass on a digit you choose is drawn on a 3D stage, with the residual stream on a spine, each block's two convolutions built out beside it, identity shortcuts as rails and the two option A shortcuts showing their zero-padded channels. A 27-step article walks through the degradation problem, the residual reformulation, 3×3 convolution, batch normalisation, the addition and its ReLU, downsampling, pooling and the classifier, the shortcut options, the bottleneck and FLOP budget, the training recipe, the clean-path and ensemble explanations, and the results, with the paper's equations and every figure hover-linked to its sentence. A pre-registered research desk then trains 24 plain and residual networks to test the degradation problem at miniature scale, re-runs Veit et al.'s lesion study on the original post-activation block, and replicates the paper's Fig. 7.
The machine
A tall tower on a table, read bottom to top like the paper's Fig. 3. The digit and its mean-subtracted input sit at the bottom, then the first convolution. Up the left runs the residual stream: each block's output, a row of channel tiles that halves in size and doubles in channels at each stage. To its right, each of the nine blocks is built out in two columns, conv1 and conv2, every 3×3 kernel bank drawn under the four batch-norm rows (γ, β, μ, σ) and the map it produces; an orange link returns F(x) to the spine. Violet rails beside the spine are the identity shortcuts; at blocks 4 and 7 an orange rail passes through the option A shortcut, drawn as its own map with its zero channels visible. At the top, global average pooling, the 32×10 classifier and the ten probabilities. Cell colour and height are the values; stage 3's kernel banks are visibly most of the weights.
What is real. Everything on the stage is computed in the page from trained weights: a 67,906-parameter ResNet-20 and its plain twin with the paper's architecture (6n+2 layers, 3×3 convolutions, BN after every convolution, option A shortcuts, global average pooling) and recipe (SGD 0.1 with momentum 0.9 and weight decay 1e-4, batch 128, learning rate divided by 10 at 50% and 75% of 164 epochs, He initialisation, pad-and-crop augmentation), trained in PyTorch on a CPU. The JavaScript forward pass matches PyTorch to 3.5e-6 on reference digits, and its held-out error and every single-block lesion match PyTorch exactly (test.mjs, 80 checks); the same tests recompute the paper's Table 6 parameter counts, the 1.8 and 3.6 billion FLOPs of its ImageNet nets and VGG-19's 19.6 billion, Veit et al.'s path-length arithmetic and every number the text states by hand. Paper figures are quoted from the text of the arXiv v1 PDF and listed in the page's fact ledger; all 103 quotes were checked mechanically against the fetched sources. The research-desk experiments, their pre-registration and results are in research/.
Working model · 3D
AI & ML · Neural network foundations
