Flashcards
2,671 cards in 36 decks, written from the same source as the concepts. Space to flip, arrows to move, K and R to sort what you know from what you do not. Progress lives in your browser.
05
Inference, Systems & Hardware
Where the model meets the silicon, the memory bus and the latency budget.
4decks
379cards
Inference Optimisation
KV cache, FlashAttention, speculative decoding, quantisation and continuous batching.
Accelerator Architecture
The memory wall, roofline analysis, GPU execution model, interconnects and systolic arrays.
Kernels & Compilers
CUDA, Triton, fusion, tiling, torch.compile, CUDA graphs and roofline-guided optimisation.
Serving Systems
Prompt caching, gateways and routing, token accounting, and multi-tenant isolation.