Concept library
883 concepts across 20 domains and 101 tracks. Each track is a coherent sequence — read it top to bottom or dip in wherever the gap is.
All domains
01Foundations
02Transformer Internals
03Training & Fine-Tuning
04Reinforcement Learning
05Inference, Systems & Hardware
06Applied LLM Engineering
07Reasoning, Evaluation & Safety
08Multimodal & Applications
09Classical ML & Statistical Learning
10Causal Inference & Experimentation
11Time Series & Forecasting
12Graphs, Recommenders & Structured Data
13Generative Modelling Beyond Transformers
14Efficiency, Compression & Edge AI
15Search & Information Retrieval
16Data & Feature Engineering
17MLOps & Platform Engineering
18Security, Privacy & Adversarial ML
19Governance, Risk & Responsible AI
20Human-AI Interaction, Product & Economics
17
MLOps & Platform Engineering
Everything between a notebook that works and a system that keeps working.
6tracks
30concepts
370cards
3.5hreading
Experiment Tracking & Reproducibility Run metadata, seeds and determinism, environment capture, and what it takes to rebuild a result a year later. 5 concepts · 60 cards
- 01 Environment Capture and Pinning Why a requirements file does not describe an environment, the layers below Python that also move, and the tradeoff between reproducibility and being able to patch a vulnerability.
- 02 Run Metadata and What to Record The minimum set of facts that makes a training run comparable and rebuildable a year later, why metrics are the least important part, and the discipline that makes tracking survive contact with a deadline.
- 03 Data Versioning and Content Addressing Why versioning datasets is harder than versioning code, how content addressing makes it tractable without copying, and what a dataset version has to mean to be useful.
- 04 Hyperparameter Search Hygiene Why random search beats grid search, how early-stopping schedulers change the budget calculation, and the selection bias that makes the best run's reported score an overestimate.
- 05 Seeds and the Limits of Determinism Why setting a seed does not make a training run reproducible, the specific sources of non-determinism on a GPU, and how to decide how much determinism is worth paying for.
Model Registry & Versioning Artefact lineage, promotion gates, rollback, and versioning models and their data together. 5 concepts · 60 cards
- 01 Model Cards and Intended Use What documentation a model version needs to be safely reused by someone who did not build it, why disaggregated evaluation is the substantive part, and how model cards become checkbox exercises.
- 02 The Registry as a Promotion Gate Why a model registry is a workflow rather than a storage location, what a stage transition should require, and the difference between a registry that records decisions and one that enforces them.
- 03 Model Lineage Across Retraining Why a chain of fine-tuned and distilled models makes provenance a graph rather than a record, what has to be tracked at each edge, and the failures that follow from losing it.
- 04 Rollback, Pinning and Reproducible Serving What it takes to revert a model change under incident conditions, why the model is usually not the only thing that has to revert, and the mechanisms that make a rollback fast rather than heroic.
- 05 What a Model Version Actually Contains Why weights alone are not a deployable model, the full set of artefacts that must move together, and the coupling failures that occur when one of them is versioned separately.
CI/CD for ML Testing pipelines that emit models, data-dependent tests, progressive delivery and automated retraining. 5 concepts · 62 cards
- 01 Build Once, Promote the Artefact Why rebuilding per environment reintroduces every difference the pipeline was meant to eliminate, how configuration is separated from the immutable artefact, and where ML deployments break the pattern.
- 02 Testing Pipelines That Emit Models What can be tested deterministically in an ML codebase, why model quality is not a unit test, and the layered test strategy that keeps CI fast while still catching the failures that matter.
- 03 Automated Retraining and Its Triggers When a retraining loop is worth building, the four trigger designs and what each optimises, and the failure modes that make automated retraining actively dangerous.
- 04 Data-Dependent Tests and Behavioural Suites Why aggregate metrics are a poor gate, how curated behavioural cases catch regressions that averages hide, and the discipline of turning every production failure into a permanent test.
- 05 Progressive Delivery for Models The staged rollout patterns that separate correctness validation from quality validation, why shadow deployment answers a question canary cannot, and what each stage should be watching.
ML Observability & Drift Feature and prediction monitoring, delayed labels, drift statistics, and alerting that does not cry wolf. 5 concepts · 64 cards
- 01 Prediction Logging and Traceability What to record at inference so that a question asked three months later has an answer, why the feature vector matters more than the input, and the sampling and privacy tradeoffs.
- 02 The Three Drifts and How They Differ Covariate shift, label shift and concept drift decomposed precisely, why only one of them necessarily degrades a model, and which of them your monitoring can actually see.
- 03 Drift Statistics and What They Miss PSI, KL divergence, KS and MMD compared on what they detect and where they fail, why per-feature tests miss joint shifts, and the multiple-comparison problem that makes wide monitoring noisy.
- 04 Monitoring Without Labels What to watch when ground truth arrives months late or never, why prediction distributions and confidence are the highest-value proxies, and how to estimate performance from unlabelled data.
- 05 Observability for LLM Applications Why the classical monitoring stack does not transfer to systems with free-text output, what a trace over an agent must capture, and the online quality signals that work without ground truth.
GPU Fleet & Capacity Scheduling and quota, fragmentation, preemption, multi-tenancy, and planning capacity under lumpy demand. 5 concepts · 62 cards
- 01 Capacity Planning Under Lumpy Demand Why GPU demand does not smooth the way CPU demand does, how to reason about the reserved-versus-on-demand mix, and the lead times that make this a quarters-ahead decision.
- 02 Fault Tolerance for Long Training Runs Why failure is the expected case at scale, the arithmetic that sets checkpoint frequency, and the detection problem that makes silent corruption worse than a crash.
- 03 Gang Scheduling and Fragmentation Why a distributed training job cannot start until every worker starts, how that requirement produces both deadlock and stranded capacity, and the placement constraints that make a cluster's usable size smaller than its size.
- 04 Multi-Tenancy and GPU Sharing Why one job per GPU wastes most of the hardware for small models, the three sharing mechanisms and their isolation guarantees, and the interference that makes sharing unsuitable for latency-sensitive work.
- 05 Preemption, Priority and Spot Capacity How interruptible capacity changes the cost of training by a large factor, what a job must be able to do to use it, and the failure modes that make spot capacity a false economy for the unprepared.
Cost & FinOps for AI Unit economics per request, token accounting, reserved versus spot capacity, and attributing spend to features. 5 concepts · 62 cards
- 01 Reserved, On-Demand and the Shape of Commitment How to choose a commitment mix when demand is uncertain, why the break-even is simply a price ratio, and the option value that makes shorter commitments rational despite costing more.
- 02 Spend Guardrails and Quotas Why AI spend can rise by orders of magnitude in hours, the layered controls that bound it without blocking legitimate work, and the design of a kill switch that is actually usable.
- 03 Unit Economics of an AI Feature How to build a cost-per-request figure that survives scrutiny, why the marginal cost of an LLM feature does not fall with scale the way software's does, and the retry and failure multipliers everyone forgets.
- 04 Token Accounting and Cost Attribution Why a single provider invoice cannot be allocated to teams, features or customers without instrumentation, and the tagging discipline that makes AI spend attributable.
- 05 Training Cost Estimation Before You Commit The FLOP arithmetic that converts a model and dataset size into a GPU-hour figure, why achieved utilisation is the term that decides the answer, and the overheads that turn an estimate into a budget.