AI Assurance & Audit
Third-party evaluation, red-team evidence, incident reporting, and control testing for AI systems.
5concepts
58flashcards
35minutes of reading
- 01 Assurance for Continuously Changing Systems Why point-in-time assurance is a poor fit for systems that retrain weekly, what continuous assurance requires instead, and how to define the change that resets the conclusion.
- 02 Control Testing for AI Systems How to test whether a stated control actually operates, the difference between design and operating effectiveness, and the AI-specific controls whose testing is unfamiliar to conventional auditors.
- 03 Independent Evaluation and Structured Access Why external scrutiny requires access that providers have reasons to withhold, the mechanisms proposed to reconcile the two, and what safe harbour would need to cover.
- 04 Red-Team Evidence and Its Limits What a red-team exercise contributes to an assurance case, why coverage cannot be quantified, and how to report results so they inform a decision rather than reassuring the reader.
- 05 What an AI Audit Can and Cannot Establish The three things an audit might mean, why access level determines what conclusions are available, and the gap between certifying a process and certifying an outcome.