Model Inventory

Every model risk control — validation, monitoring, documentation, tiering, revalidation — applies to a model the organisation knows about. An unregistered model is ungoverned by construction.

Most organisations cannot list their models, and the ones they can list are the ones built by the teams already following the process.

What the inventory must include

• Production machine learning systems, which is the easy and usually complete part.
• Models in notebooks and spreadsheets informing real decisions. These are the ungoverned majority in many organisations, and excluding them makes the inventory a register of the already-compliant.
• Third-party and vendor models, where the organisation carries the risk and does not control the model.
• Rule-based scoring systems with the same consequence profile — the risk attaches to the decision, not to the technique, and a simple scoring rule denying credit is a higher-risk system than a large model writing marketing copy.
• Purpose, owner, tier, validation status, and the decision each one drives.

Implementation patterns

• Automated discovery where possible — model registries, serving infrastructure, feature store usage — so the inventory is not solely dependent on self-declaration.
• Registration as a deployment gate, so a model cannot reach production unregistered.
• Tier assigned at registration, determining the controls automatically rather than through negotiation.
• Linked to lineage: data, code, configuration and version, so a production model traces to what produced it.
• Periodic attestation by owners, which catches decommissioned models still registered and, more usefully, prompts disclosure of undeclared ones.
• Fallback behaviour recorded, so it is known what happens when each model is unavailable or untrusted.

Industry example

Financial institutions were required to maintain these long before machine learning was widespread, and the recurring finding in every industry that follows is the same: the inventory discovers substantially more models than expected, and the highest-risk ones are frequently the least sophisticated — a spreadsheet driving pricing decisions, a rules engine determining eligibility, a vendor score nobody has validated.

The complementary lesson is that training and serving must share feature definitions. Divergence produces silent quality loss that presents as data drift and is actually a definition mismatch — which is only diagnosable if the inventory records how each model's features are computed on both paths.

Failure scenarios

• Self-declaration only, producing a register of the already-compliant.
• Production ML only, missing spreadsheets, rules engines and vendor models.
• Tiering by model sophistication rather than by decision consequence.
• A one-time exercise, accurate on the day it was compiled.
• No link to lineage, so a registered model cannot be traced to the data and code that produced it.

Trade-offs

A comprehensive inventory is intrusive to compile and generates governance obligations proportional to its completeness — which creates a genuine incentive not to look too hard, and is worth naming when the programme is designed.

Proportionality is what makes it sustainable: a low tier should carry a genuinely light obligation — an owner, a purpose, basic monitoring — so that registering a minor model is cheap. An inventory whose every entry triggers a heavy process will be incomplete by design.

Interview question

"How many models does your organisation have in production, and how would you find out? Tell me which categories your first answer would miss."