Mathematical Foundations
Linear algebra, probability, calculus and optimisation — the machinery every model is built on.
16concepts
223flashcards
140minutes of reading
- 01 Calculus and Gradients Partial derivatives, the chain rule as the engine of backprop, why second-order methods are rare in deep learning, and what gradient clipping actually does.
- 02 Fourier Analysis for Neural Models How the convolution theorem turns quadratic sequence mixing into n log n, why MLPs refuse to learn high frequencies, and why rotary position encoding is a frequency argument wearing a rotation costume.
- 03 Linear Algebra for ML Vector spaces, matrix decompositions, and why low-rank structure underlies LoRA, PCA, and the quantisation tricks that make modern LLMs cheap to serve.
- 04 Numerical Computation Gotchas Catastrophic cancellation, the log-sum-exp trick, mixed-precision training, the determinism tax, and how to actually debug a NaN in a 70B model.
- 05 Probability and Information Theory Distributions, expectations, entropy, KL, and why softmax + cross-entropy is the canonical pair that secretly underlies almost every LLM loss.