Variational & Flow Models
The ELBO, reparameterisation, normalising flows, flow matching and rectified transport.
5concepts
64flashcards
37minutes of reading
- 01 The Reparameterisation Trick Why you cannot backpropagate through a sampling operation, how moving the randomness to an input fixes it, and the variance argument that explains why this beats the score-function estimator.
- 02 Flow Matching and Conditional Vector Fields How to train a continuous-time generative model by regressing a velocity field without ever simulating an ODE, and why conditioning on a single data point makes an intractable target tractable.
- 03 Normalising Flows and the Cost of Invertibility How change of variables turns a simple density into a complex one with an exact likelihood, why the Jacobian determinant constrains every architectural choice, and what that constraint costs relative to VAEs and diffusion.
- 04 Posterior Collapse and the KL Term The failure where a VAE's latent code carries no information, why a powerful decoder makes it the optimal solution rather than an accident, and the fixes that each buy something different.
- 05 Rectified Flow and Trajectory Straightening Why the generative ODE learned from independent noise-data pairs is curved, how reflow iteratively straightens it, and what each round of straightening costs in fidelity.