Classical Supervised Learning
Linear and logistic regression, regularisation, margins, kernels, and the geometry underneath them.
8concepts
156flashcards
60minutes of reading
- 01 Linear Regression as Projection Least squares is orthogonal projection onto the column space of the design matrix, which explains the normal equations, the meaning of residuals, and why nobody who ships numerical code inverts the matrix.
- 02 Logistic Regression and the Log-Odds Why classification is modelled on the log-odds scale rather than the probability scale, why there is no closed-form solution, and what perfect separation does to the coefficients.
- 03 k-Nearest Neighbours and the Curse of Dimensionality The simplest classifier there is comes with a strong guarantee in low dimensions and a quiet collapse in high ones, because neighbourhoods stop being local and distances stop discriminating.