Mathematical Foundations
Linear algebra, probability, calculus and optimisation — the machinery every model is built on.
6concepts
30flashcards
54minutes of reading
- 01 Calculus and Gradients Partial derivatives, the chain rule as the engine of backprop, why second-order methods are rare in deep learning, and what gradient clipping actually does.
- 02 Linear Algebra for ML Vector spaces, matrix decompositions, and why low-rank structure underlies LoRA, PCA, and the quantisation tricks that make modern LLMs cheap to serve.
- 03 Numerical Computation Gotchas Catastrophic cancellation, the log-sum-exp trick, mixed-precision training, the determinism tax, and how to actually debug a NaN in a 70B model.
- 04 Probability and Information Theory Distributions, expectations, entropy, KL, and why softmax + cross-entropy is the canonical pair that secretly underlies almost every LLM loss.