Efficient Architectures
Small language models, depth-width tradeoffs, weight sharing, and architectures designed for a latency budget.
5concepts
58flashcards
36minutes of reading
- 01 Architecting to a Latency Budget How to design a model backwards from a millisecond target using arithmetic intensity, why parameter count is the wrong currency, and the design moves that actually reduce time to first and subsequent tokens.
- 02 Depth Versus Width Tradeoffs Why two models with identical parameter counts behave differently depending on how the parameters are arranged, what depth buys that width cannot, and how the hardware votes for width.
- 03 Hardware-Aware Architecture Search Why optimising a model for FLOPs produces slow models, how measured latency became the objective instead, and what makes once-for-all supernet training the practical form of the idea.