Synthetic Data
Self-Instruct, distillation, self-play, and how to generate training data without collapsing.
20concepts
140flashcards
150minutes of reading
- 01 Why Synthetic Data Synthetic data lets practitioners generate arbitrarily large, precisely labelled training sets without manual annotation, but only when the generator is accurate enough not to amplify its own errors.
- 02 Auditing Synthetic Data Auditing synthetic training data means applying measurable quality checks, coverage tests, and contamination scans before the data ever touches a training run.
- 03 Distillation and Terms-of-Service Constraints Using a commercial API to generate training data for a competing model almost universally violates the provider's terms of service, and understanding exactly why - and what compliant alternatives exist - is non-negotiable before building any distillation pipeline.
- 04 Distilling Reasoning Traces Reasoning-trace distillation transfers step-by-step chain-of-thought outputs from a large teacher model into fine-tuning data for a smaller student, giving the student reasoning capability it could not develop on final-answer supervision alone.
- 05 Diversity Metrics and Collapse Detection Quantitative metrics for detecting when synthetic training data loses coverage of the original distribution, with practical monitoring strategies to catch collapse before it harms the next model generation.
- 06 Evol-Instruct Evol-Instruct is an LLM-driven pipeline that iteratively rewrites seed instructions into progressively harder variants, enabling automated construction of complex instruction-following training data without human labellers.
- 07 Knowledge Distillation from a Teacher Model Knowledge distillation trains a smaller student model to reproduce a larger teacher's output distribution, enabling compact models with performance well beyond what their size alone would predict.
- 08 Mixing Synthetic and Human Data Balancing synthetic and human-authored training examples determines whether a model inherits the strengths of both sources or the worst of each.
- 09 Persona Prompting for Diversity Persona prompting injects fictional user identities into an LLM's prompt to steer it toward generating training data that spans a broader slice of the real distribution than naive repeated sampling achieves.
- 10 Programmatic and Templated Data Generation Programmatic and templated generation produces synthetic training data through deterministic code, slot-filling templates, and context-free grammars, giving precise control over distribution and format that purely model-driven pipelines cannot match.
- 11 Quality Filtering of Synthetic Data Quality filtering removes low-signal, redundant, or contaminated examples from synthetically generated datasets before fine-tuning, preventing reward hacking and model collapse.
- 12 Rejection Sampling Fine-Tuning and STaR Rejection sampling fine-tuning and the STaR algorithm are iterative self-improvement techniques that generate synthetic training data by having a model solve problems, keeping only the correct solutions, and re-training on those filtered traces.
- 13 Self-Instruct Self-Instruct is a bootstrapping pipeline that uses a language model's own outputs to generate large-scale instruction-following training data with minimal human annotation.
- 14 Self-Play and Self-Improvement Self-play loops use a model to generate, critique, and filter its own training data, compounding capability without proportionally scaling human annotation effort.
- 15 Synthetic Data for Code Generating synthetic code training data via instruction synthesis, distillation, and execution-based filtering lets small models punch well above their weight, but only when a reliable verifier anchors the loop.
- 16 Synthetic Data for Mathematics Synthetic math data pipelines use teacher models, rejection sampling, and question rewriting to bootstrap training corpora far larger than any human-labelled set, but the quality ceiling is set by the generator's own reasoning ability.
- 17 Synthetic Preference Data Synthetic preference data replaces expensive human comparison labels by having a language model judge which of two candidate responses is better, enabling scalable RLHF-style alignment without a large human labelling workforce.
- 18 Textbook-Quality Synthetic Data How to generate training data that teaches models to reason rather than memorise, using instruction synthesis, distillation, rejection sampling, and constitutional loops, along with the collapse risks that follow.
- 19 Model Collapse from Recursive Training When a model is trained repeatedly on its own outputs, tail distributions erode and the model progressively forgets rare but important knowledge, eventually producing impoverished, homogenised text.
- 20 The Constitutional AI Data Loop Constitutional AI replaces most human preference labels with a self-critique-and-revise loop guided by a written list of principles, producing both supervised fine-tuning data and AI-labelled preference pairs that train a reward model.