practice

Model Card

A structured record of what a model is for, how it was built and where it should not be used — written for the people who will deploy or be affected by it.

A model without documentation of its intended use will be used outside it. A credit model trained on one population is applied to another; a classifier validated on one image distribution is deployed on a different camera; a language model evaluated on English is used in another language.

A model card states, in a fixed structure: the intended use and the users; the training data and its provenance; evaluation results disaggregated by relevant subgroup, since aggregate accuracy can hide substantially worse performance on a minority group; known limitations and out-of-scope uses; ethical considerations; and the maintenance and retraining arrangements.

The disaggregation is the part that carries the most information and is most often omitted. A model reported at ninety-two percent accuracy may be at ninety-six for one group and seventy-one for another, and the aggregate figure conceals exactly the problem that matters.

The section that prevents the most harm is out-of-scope uses, stated explicitly rather than implied. It gives a downstream team a clear answer, and it gives the model's owners something to point at when a proposed use is inappropriate.

Increasingly this is a regulatory expectation rather than good practice, and the work is substantially cheaper done during development than reconstructed afterwards.