Model Cards and Intended Use

A model in a registry will be used by people who did not build it, in contexts its authors did not anticipate. Model cards (Mitchell et al., 2019, Model Cards for Model Reporting, FAccT) exist to make the boundary of appropriate use explicit, and the parts that carry the weight are the ones teams most often skip.

The sections that matter

Intended use, and out-of-scope use. The second is the more useful. A credit risk model evaluated on applicants aged 25 to 65 should say that it was not evaluated outside that range, so a team applying it to a younger population knows they are extrapolating. Stating what a model is not for prevents more harm than stating what it is for.

Disaggregated evaluation. Aggregate accuracy hides subgroup failure. A model at 94 percent overall can be at 71 percent for a subgroup that is 5 percent of the data, and that difference is invisible in any headline number. Reporting performance broken down by the axes that matter for the application, with confidence intervals reflecting the smaller sample sizes, is the substantive content of a model card and the part most often reduced to a single table of overall metrics.

Training data description. Sources, time period, size, and known gaps. The time period matters more than teams expect, since it bounds what the model can know and predicts when it will go stale.

Limitations and failure modes. Concrete, observed failures rather than generic caveats. "Degrades on inputs longer than 2,000 tokens" is useful. "May produce incorrect results" is not.

Ethical and safety considerations where applicable, including how the model behaves on sensitive inputs and what mitigations exist.

Why they degrade into checkboxes

Model cards become perfunctory for structural reasons rather than through negligence. They are written at the end of a project by whoever has least context. Nobody reads them, so no feedback improves them. The genuinely useful content, disaggregated evaluation and observed failure modes, requires work that is not required to ship. And a template with headings invites filling in headings rather than answering questions.

The counters are specific: generate the mechanical sections automatically from the registry, since metrics, data versions and training configuration are all recorded already; require disaggregated results as a promotion gate rather than as documentation; and write limitations from actual observed failures collected during evaluation, which means collecting them.

When it breaks

Documentation does not constrain use. A card saying a model is out of scope for a population does not stop anyone using it there. Where the constraint matters, it belongs in the serving layer as a validation rule, and the card explains the rule rather than substituting for it.

Cards go stale. A model retrained monthly needs its card regenerated monthly, which only happens if generation is automated. A card describing the training data of a version eleven revisions ago is worse than none, since it is authoritative-looking and wrong.

Disaggregation requires attributes you may not have. Reporting performance by demographic group requires knowing group membership, which is often deliberately not collected. Proxy methods exist and are error-prone, and the honest card states that the evaluation could not be disaggregated and why, rather than omitting the section.

Regulatory documentation is a different artefact. Technical documentation required under regimes such as the EU AI Act has prescribed content and an audience of assessors. A model card is a useful input to it and is not a substitute, and teams that assume otherwise discover the gap late.