Decoding & Generation
Greedy, beam, temperature, top-k, nucleus, and constrained generation into structured formats.
12concepts
51flashcards
89minutes of reading
- 01 Greedy Decoding The simplest possible decoder, take the single most probable token at every step, and why that local optimality guarantees nothing about the sentence you end up with.
- 02 Temperature Sampling Dividing logits by a scalar before softmax to sharpen or flatten a model's output distribution, and why temperature alone is a poor substitute for a real safety mechanism.
- 03 Top-k Sampling Truncating the vocabulary to a fixed-size shortlist of the k most probable tokens before sampling, and why a fixed k is systematically the wrong size for at least some fraction of every distribution.