Safety & Alignment
Prompt injection, jailbreaks, Constitutional AI, reward hacking and mechanistic interpretability.
11concepts
54flashcards
93minutes of reading
- 01 Alignment Evaluations and Frontier-Model Risk How frontier labs and governments measure dangerous capabilities, what an eval-gated release looks like, and where the regulatory regime sits in 2026.
- 02 Jailbreaks and Refusal Robustness How attackers reliably bypass model refusal training, why post-hoc filters are necessary but never sufficient, and how AILuminate measures what remains.
- 03 Prompt Injection Why LLMs cannot reliably tell instructions from data, how indirect injection weaponises retrieved content, and which partial defences are worth deploying.
- 04 Watermarking and Content Provenance How a statistical signal is embedded in generated text without changing its quality, why SynthID-Text scaled to production, and the robustness limits every scheme shares.