Training Objectives
Next-token prediction, masked LM, span corruption, fill-in-the-middle, and auxiliary losses.
13concepts
66flashcards
91minutes of reading
- 01 Next-Token Prediction and Cross-Entropy Loss The single objective behind every LLM - predict the next token - and the cross-entropy loss that turns "predict well" into a gradient the model can descend.
- 02 Softmax and Logits How a model's raw output scores become a probability distribution, why the exponential matters, and the numerical trick that keeps softmax from overflowing.
- 03 The Causal LM Pretraining Task How a corpus of raw documents becomes millions of training examples under one masking rule, and why "causal" describes a data and attention decision, not just a loss.
- 04 Fill-in-the-Middle A way to teach a plain causal decoder to infill text by rearranging training documents, not the model, so a single architecture serves both left-to-right generation and code-editor-style completion.
- 05 Label Smoothing Softening a one-hot target so the model is never rewarded for driving a correct-class probability all the way to 1, trading a little training loss for calibration and generalisation.
- 06 Loss Masking in Fine-Tuning Supervised fine-tuning uses the same shifted next-token loss as pretraining, computed over the exact same kind of sequence; the only change is that most of the sequence is excluded from the loss entirely.
- 07 Masked Language Modelling BERT's pretraining task hides tokens instead of futures, trading the ability to generate text for representations that read both directions at once.
- 08 Pretraining Objectives Why the loss a model is trained to minimise decides what it can become, and how next-token prediction beat masked and span objectives to own the generative era.
- 09 Sequence Packing and Document Masking Concatenating short documents to fill a fixed context length eliminates padding waste, but only if the attention mask and position IDs are made document-aware; done naively, packing quietly teaches the model to attend across unrelated documents.
- 10 Span Corruption T5's pretraining task hides whole spans instead of single tokens and asks a decoder to generate only the missing pieces, trading BERT's per-token reconstruction for a shorter, cheaper target sequence.
- 11 MoE Load-Balancing Loss Without an explicit penalty, a mixture-of-experts router collapses onto a handful of favourite experts within the first few hundred steps; the auxiliary load-balancing loss is the mechanism that stops it.
- 12 The Softmax Cross-Entropy Gradient The combined derivative of softmax and cross-entropy collapses to one subtraction, predicted probability minus true label, and that simplification is why every production training loop fuses the two ops instead of computing them separately.
- 13 Z-Loss and Logit Regularisation Softmax is shift-invariant, which leaves a direction in logit space that cross-entropy never penalises; z-loss closes that gap and is what keeps large-scale bf16 training from spiking.