Unsupervised Learning
Clustering, mixture models and EM, PCA and SVD, manifold embeddings, and density estimation.
7concepts
145flashcards
53minutes of reading
- 01 K-Means and Its Assumptions Lloyd's algorithm is coordinate descent on a squared-error objective, which explains why it always converges, why it converges to the wrong answer without careful seeding, and the exact cluster shapes it cannot represent.
- 02 Density-Based Clustering with DBSCAN and HDBSCAN DBSCAN defines a cluster as a connected region of high point density and labels everything else as noise, and HDBSCAN removes its single density threshold by building the whole hierarchy of thresholds and keeping the most persistent clusters.
- 03 Hierarchical Clustering and Linkage Agglomerative clustering builds a full tree of merges instead of one partition, and the linkage rule that scores each merge decides whether the tree finds crescents, chains through noise, or reproduces k-means.
- 04 PCA, SVD and Whitening Why maximising retained variance and minimising reconstruction error give the same answer, how the SVD computes it without ever forming a covariance matrix, and what whitening destroys.
- 05 UMAP, t-SNE and What They Distort Neighbour embeddings optimise local neighbourhood preservation and nothing else, which makes cluster separation, cluster size and inter-cluster distance in the resulting picture largely uninterpretable.
- 06 Clustering Validation: Silhouette, Gap, ARI and Choosing k Internal indices like silhouette and the gap statistic score a clustering against its own geometric ideal, external indices like ARI and NMI need chance correction to mean anything, and Kleinberg's impossibility theorem explains why no index can settle what the right clustering is.
- 07 Gaussian Mixtures and the EM Algorithm How treating the cluster label as a latent variable turns an intractable likelihood into two closed-form steps, why the likelihood is guaranteed to increase, and why it is unbounded above.