A model makes decisions affecting people. What documentation is actually required, and who is it for?
Show the full answer Hide the answer
Who it is for
Three audiences with different needs, and documentation written for one serves the others badly:
- The people operating it, who need to know what it does, its known failure modes, and when to escalate.
- The people affected by it, who need an explanation of a decision and a route to challenge it.
- The assurance function and the regulator, who need evidence that the risk was assessed and the controls operate.
What must be documented
- The purpose and the scope, precisely — including what it is not suitable for, which is the section that allows a downstream user to make their own assessment and whose absence makes that impossible.
- The data used, its provenance, and its known limitations and biases.
- Evaluation results with subgroup breakdown, since aggregate performance can be excellent while a subgroup is served badly and that subgroup is frequently the one a regulator cares about.
- The human role, stated honestly. A human who approves ninety-nine percent of recommendations without independent evidence is not a control, and describing them as one is the most common misrepresentation in these documents.
- The monitoring in place, and what would trigger a review.
- A named accountable owner, since a model with no owner is one nobody can be asked about.
What the affected person needs
An explanation that is meaningful rather than technically complete. The main factors, expressed in terms they can act on, and a route to challenge that reaches a human with the authority to change the outcome.
An appeal that returns the same automated answer is not an appeal, and it is the design failure regulators increasingly examine.
The documentation that is genuinely load-bearing
The stated limitations. They are what allows a customer, a regulator or an internal user to assess whether the model is appropriate for their case — and an over-claiming document transfers a risk the organisation cannot control to a user who cannot assess it.
Honest limitations are commercially uncomfortable and are the section with the most practical value.