Flashcards

11,105 cards in 101 decks, written from the same source as the concepts. Space to flip, arrows to move, K and R to sort what you know from what you do not. Progress lives in your browser.

All decks 01Foundations 02Transformer Internals 03Training & Fine-Tuning 04Reinforcement Learning 05Inference, Systems & Hardware 06Applied LLM Engineering 07Reasoning, Evaluation & Safety 08Multimodal & Applications 09Classical ML & Statistical Learning 10Causal Inference & Experimentation 11Time Series & Forecasting 12Graphs, Recommenders & Structured Data 13Generative Modelling Beyond Transformers 14Efficiency, Compression & Edge AI 15Search & Information Retrieval 16Data & Feature Engineering 17MLOps & Platform Engineering 18Security, Privacy & Adversarial ML 19Governance, Risk & Responsible AI 20Human-AI Interaction, Product & Economics

Training & Fine-Tuning

From raw web crawl to an aligned model — data, dynamics, scale and adaptation.

6decks

886cards

Pretraining Data Pipelines

Web-scale corpus construction, filtering, deduplication, decontamination and data mixtures.

140 cards 20 concepts

Synthetic Data

Self-Instruct, distillation, self-play, and how to generate training data without collapsing.

140 cards 20 concepts

Training Dynamics & Scaling

Learning-rate schedules, warmup, loss spikes, critical batch size, muP, and scaling laws.

157 cards 16 concepts

Distributed Training

Data, tensor and pipeline parallelism, ZeRO/FSDP sharding, mixed precision, and offload.

165 cards 13 concepts

Parameter-Efficient Fine-Tuning

LoRA, QLoRA, DoRA, adapters and soft prompts — adapting big models on small budgets.

140 cards 20 concepts

Alignment & Post-Training

SFT, reward modelling, DPO/IPO/KTO/ORPO, model merging, and evaluating an aligned model.

144 cards 21 concepts