Tensors & Neural Plumbing
Shapes, matmuls, forward and backward passes, parameter counts, memory footprints.
18concepts
211flashcards
138minutes of reading
- 01 Broadcasting and Vectorisation Broadcasting is the rule that lets a bias vector add to every row of a batch without an explicit loop, and understanding its shape-matching logic prevents the class of bugs that produce wrong answers without an error.
- 02 Matrix Multiplication: The Core Operation Nearly every FLOP a transformer spends is a matrix multiplication; understanding its shape rule and its cost is the single most load-bearing piece of maths in deep learning.
- 03 Tensors, Shapes, and Batching Every number a transformer touches lives inside a tensor with a fixed shape; learning to read and predict those shapes is the fastest way to stop being confused by model code.
- 04 Why Non-Linearity Matters Stack any number of linear layers and the result is still one linear layer; the humble activation function is the only thing standing between a transformer and a glorified matrix multiplication.