Deep Learning Building Blocks
Convolutions, recurrence, normalisation, activations, optimisers and regularisation.
13concepts
112flashcards
102minutes of reading
- 01 State Space Models and Selective SSMs How a linear recurrence with a structured state matrix reaches transformer-level language modelling at linear cost and constant-memory decoding, what selectivity added, and the copying tasks that still expose the gap.
- 02 Variational Autoencoders and the ELBO How maximising an intractable data likelihood turns into maximising a tractable lower bound, why the reparameterisation trick is what makes that bound differentiable, and what posterior collapse costs you.