What should model documentation contain, and who is it actually for?

Who reads it

Three audiences with different needs, and documentation written for one serves the others poorly:

• Someone deciding whether to use the model for a new purpose. They need intended use, limitations, performance by population, and what it must not be used for.
• Someone validating or auditing it. They need methodology, data provenance, assumptions, evaluation design and known weaknesses.
• Someone operating it. They need the monitoring signals, the retraining triggers, the fallback behaviour and the escalation path.

What it must contain

• Intended purpose and, explicitly, out-of-scope uses. Reuse outside the intended context is one of the most common ways models cause harm, and the documentation is the only control that addresses it.
• Training data provenance, including collection purpose and known gaps — which is also what makes a purpose-limitation question answerable later.
• Performance segmented by population, not only in aggregate, since aggregate performance hides exactly the disparities that matter.
• Known limitations and failure modes, stated plainly rather than minimised.
• Evaluation methodology, so results can be interpreted and reproduced.
• Version and lineage, linking the model to its data, code and configuration.

Why it decays

Documentation written once at release and never updated diverges from a model that has been retrained several times. Generating what can be generated — performance metrics, data lineage, version linkage — from the pipeline keeps the factual parts current, and leaves humans to write the parts that require judgement: intended use, limitations, and what it must not be used for.

The test

Could someone outside the team decide correctly whether to use this model for a new purpose? If not, the documentation has failed at the thing it is most needed for.