Attention Internals
Queries, keys and values, masking, multi-head and grouped-query, sinks, and the quadratic wall.
11concepts
49flashcards
88minutes of reading
- 01 Attention Sinks Why the first few tokens of almost any sequence soak up a disproportionate share of attention regardless of their content, and how that quirk becomes the key to stable infinite-length streaming generation.
- 02 Induction Heads A specific, mechanistically understood attention circuit that copies patterns it has seen once in the current context, and the closest thing the field has to a concrete explanation for in-context learning.
- 03 Multi-Query and Grouped-Query Attention Why the key and value projections, not the query projection, were the ones cut down to fix the real inference bottleneck, and the middle-ground design that most production LLMs settled on.