Sparsity & Pruning
Magnitude and second-order criteria, structured versus unstructured sparsity, and the hardware that rewards it.
5concepts
62flashcards
36minutes of reading
- 01 Magnitude Pruning and the Lottery Ticket Hypothesis Why the simplest possible pruning criterion is so hard to beat, what the lottery ticket experiment actually claims, and the rewinding detail that decides whether it reproduces.
- 02 Structured Versus Unstructured Sparsity Why 90 percent unstructured sparsity can be slower than a dense matmul, what removing a whole channel buys instead, and how to decide which side of the tradeoff a deployment sits on.