Training & Alignment
27 min
What the Model Remembers: Extraction, Memorisation, and the Price of a Privacy Guarantee
Two hundred dollars of API calls pulled more than ten thousand verbatim training examples out of ChatGPT. Memorisation is not a defect that better engineering removes; it scales log-linearly with everything the field is scaling. …