Sparse Autoencoders for Feature Extraction

Pick a neuron halfway through a language model and plot what makes it fire. You get the Golden Gate Bridge, a Python decorator, the Cyrillic letter ж, and the word "however" at the start of a clause. The neuron is not confused; the model is storing far more distinct concepts than it has dimensions, packed as overlapping directions in the residual stream. That is superposition, and it makes neuron-by-neuron interpretation a dead end. A sparse autoencoder (SAE) is the standard tool for undoing the packing: instead of asking what a neuron means, it learns a much larger dictionary of directions and insists that only a handful of them are active on any one token.

Dictionary learning on activations

Take an activation vector \(x \in \mathbb{R}^{d}\) from some site in the model, usually the residual stream at a middle layer. An SAE encodes it into a much wider latent vector and decodes back:

The dictionary \(W_{dec}\) has \(m \gg d\) columns, each column a candidate feature direction. Expansion factors of 8× to 100× are routine, and production runs go much wider: Anthropic trained 1M, 4M and 34M-feature SAEs on Claude 3 Sonnet (Templeton et al., 2024, Scaling Monosemanticity), and OpenAI trained a 16M-latent SAE on GPT-4 activations over 40B tokens (Gao et al., 2024, arXiv:2406.04093).

Training is reconstruction plus a sparsity pressure:

Two forces in tension. Reconstruction wants to keep every bit of \(x\); the sparsity term wants almost all of \(f(x)\) to be zero. The useful regime is a few dozen active features out of millions. Because an L1 penalty shrinks the activations it is trying to select, later variants replace it: k-sparse autoencoders take the top \(k\) latents and zero the rest, which sets sparsity directly instead of tuning \(\lambda\) (Gao et al., 2024), and JumpReLU SAEs use a discontinuous threshold with straight-through gradients for better reconstruction at matched sparsity (Rajamanoharan et al., 2024, arXiv:2407.14435).

graph LR
  A["residual stream x (d ≈ 4k)"] --> B["encoder + sparsity"]
  B --> C["f(x): millions of latents,<br/>tens non-zero"]
  C --> D["decoder W_dec"]
  D --> E["x̂ ≈ x"]
  C --> F["label features<br/>from max-activating text"]
  C --> G["clamp a feature<br/>→ steer behaviour"]

Labelling and steering

A trained SAE gives you directions, not meanings. Meaning comes from looking: collect the text that maximally activates each latent, and have a model write a description of the pattern. Feature 34M/31164353 in the Sonnet SAE fires on the Golden Gate Bridge, including Wikipedia descriptions in Chinese, Japanese, Korean and Greek, which is stronger evidence of an abstract concept than any English-only correlation would be.

Correlation is where most interpretability claims stop. The reason SAEs are taken seriously is the causal follow-up: clamp a feature to a large multiple of its observed maximum and rerun the forward pass. Clamping the bridge feature to 10× made Claude 3 Sonnet describe itself as the Golden Gate Bridge. Steering also works on safety-relevant features, which is why this line of work sits next to alignment rather than inside pure interpretability.

Not a probe, not a neuron

A linear probe is supervised: you already have labels for "is this deceptive", you fit a direction, and you get a good classifier for that one thing. An SAE is unsupervised: it proposes a whole vocabulary of features before anyone specifies what to look for, which is the only way to find concepts you did not think to ask about. The price is that nothing guarantees the vocabulary carves the model at its joints, and for any concept you can label, a probe is usually the stronger classifier.

When it breaks

Reconstruction is never complete. The Sonnet SAEs explained at least 65% of activation variance, meaning a third of the signal is thrown away, and splicing \(\hat{x}\) back into the forward pass measurably degrades the model. Any story told from features is a story about the part that reconstructed.

Widening the dictionary buys resolution and costs yield. Dead latents in the Sonnet runs went from roughly 2% at 1M features to 65% at 34M: two thirds of the dictionary never fired at all. Wider SAEs also split concepts, so one "legal document" feature becomes forty near-duplicates, and related features absorb each other's activations in ways that make individual latents misleading.

The deepest problem is the absence of ground truth. There is no held-out set of true features to score against, so evaluation leans on proxies, and the proxies have not been kind. On sparse probing tasks, SAE features generally failed to beat simple baselines outside a few data-scarce and label-noise regimes (Kantamneni et al., ICML 2025, arXiv:2502.16681). As of 2026 the technique is the best available handle on superposition and still an open bet, not a solved instrument.