Adversarial Robustness
Perturbation attacks, adversarial training, certified defences, and the robustness-accuracy tradeoff.
12concepts
103flashcards
93minutes of reading
- 01 Adversarial Examples and the Threat Model Why an imperceptible perturbation flips a confident classifier, what a norm ball is actually assuming, and why stating the threat model precisely is the first substantive step in any robustness claim.
- 02 Attack Scaling and the Sampling Budget Why attack success against a language model is a function of how many times the attacker is allowed to try, how many-shot and best-of-N jailbreaking turn that into a predictable power law, and what it does to any robustness number reported at a single budget.
- 03 Layered Defences and the Cost of Safeguards What a classifier layer in front of and behind a model actually buys against universal jailbreaks, measured in success rate, false refusals and compute overhead, and why cascading cheap screens before expensive ones is the engineering that made it deployable.