Context & In-Context Learning
Autoregressive generation, the prompt stack, context engineering, and long-context degradation.
6concepts
28flashcards
46minutes of reading
- 01 Context Rot The measured fact that model accuracy falls as the input grows, non-uniformly and in cliffs, so a bigger window is a bigger desk rather than a better memory.
- 02 Context Windows and Long-Context Models Why a model advertised at a million tokens can still lose the fact in the middle, and what actually sets the limit: memory, compute, position, and attention itself.