Flashcards
2,671 cards in 36 decks, written from the same source as the concepts. Space to flip, arrows to move, K and R to sort what you know from what you do not. Progress lives in your browser.
Foundations
The mathematics and neural-network mechanics everything else assumes.
Mathematical Foundations
Linear algebra, probability, calculus and optimisation — the machinery every model is built on.
Tensors & Neural Plumbing
Shapes, matmuls, forward and backward passes, parameter counts, memory footprints.
Deep Learning Building Blocks
Convolutions, recurrence, normalisation, activations, optimisers and regularisation.
Information Theory for Language
Entropy, cross-entropy, KL, perplexity, calibration, and language modelling as compression.
Transformer Internals
Open the box. How a language model actually turns text into predictions.
Tokenisation
BPE, WordPiece, Unigram, and the ways subword vocabularies quietly shape model behaviour.
Embeddings & Representations
The lookup table, the residual stream, contextual vectors, geometry and superposition.
Attention Internals
Queries, keys and values, masking, multi-head and grouped-query, sinks, and the quadratic wall.
Positional Encoding
Sinusoidal, learned, RoPE, ALiBi, and how context windows get stretched past training length.
Transformer Anatomy
The block, the stack, encoder vs decoder, MoE, and the design choices that separate model families.
Training Objectives
Next-token prediction, masked LM, span corruption, fill-in-the-middle, and auxiliary losses.
Decoding & Generation
Greedy, beam, temperature, top-k, nucleus, and constrained generation into structured formats.
Context & In-Context Learning
Autoregressive generation, the prompt stack, context engineering, and long-context degradation.
Training & Fine-Tuning
From raw web crawl to an aligned model — data, dynamics, scale and adaptation.
Pretraining Data Pipelines
Web-scale corpus construction, filtering, deduplication, decontamination and data mixtures.
Synthetic Data
Self-Instruct, distillation, self-play, and how to generate training data without collapsing.
Training Dynamics & Scaling
Learning-rate schedules, warmup, loss spikes, critical batch size, muP, and scaling laws.
Distributed Training
Data, tensor and pipeline parallelism, ZeRO/FSDP sharding, mixed precision, and offload.
Parameter-Efficient Fine-Tuning
LoRA, QLoRA, DoRA, adapters and soft prompts — adapting big models on small budgets.
Alignment & Post-Training
SFT, reward modelling, DPO/IPO/KTO/ORPO, model merging, and evaluating an aligned model.
Reinforcement Learning
Classical RL, then the specific dialect of it that post-trains language models.
Inference, Systems & Hardware
Where the model meets the silicon, the memory bus and the latency budget.
Inference Optimisation
KV cache, FlashAttention, speculative decoding, quantisation and continuous batching.
Accelerator Architecture
The memory wall, roofline analysis, GPU execution model, interconnects and systolic arrays.
Kernels & Compilers
CUDA, Triton, fusion, tiling, torch.compile, CUDA graphs and roofline-guided optimisation.
Serving Systems
Prompt caching, gateways and routing, token accounting, and multi-tenant isolation.
Applied LLM Engineering
Building things people use: retrieval, prompts, agents and production architecture.
Retrieval & RAG
Vector stores, hybrid retrieval and reranking, and when to retrieve instead of fine-tune.
Prompt Engineering
In-context learning, chain of thought, structured output, compression and injection-aware design.
Agents & Tool Use
Function calling, ReAct loops, MCP, agent memory architectures and evaluation harnesses.
Claude Certified Architect
Agentic loops, coordinator-subagent designs, tool interfaces and reliability patterns for Claude.
Reasoning, Evaluation & Safety
Models that think longer, the evals that measure them, and the failure modes that matter.
Reasoning Models
Test-time compute, process reward models, the o-series, DeepSeek-R1 and contamination.
Evaluation & MLOps
Benchmarks, LLM-as-judge, red-teaming, model registries, drift detection and observability.
Safety & Alignment
Prompt injection, jailbreaks, Constitutional AI, reward hacking and mechanistic interpretability.
Multimodal & Applications
Beyond text — vision, speech, robotics and scientific discovery.
Vision & Multimodal
ViT, CLIP, diffusion, SAM, and the vision-language models that read images as tokens.
Speech Recognition
Spectrograms, CTC, RNN-T, Conformer, Whisper, streaming, diarisation and self-supervised audio.
Speech Synthesis
Acoustic models and vocoders, Tacotron, FastSpeech, HiFi-GAN, neural codecs and voice cloning.
Robotics & Embodied AI
Vision-language-action models, action tokenisation, diffusion policies and sim-to-real.
AI for Science
AlphaFold, protein language models, materials discovery, and the pitfalls of ML-for-science.