A vendor SaaS product embeds a model that scores customers, and its output drives an automated decision in your process. Your model governance framework covers models you build. What do you do?

The obligation does not transfer with the outsourcing

You are accountable for the decision. That the scoring is performed by a vendor changes who operates the model, not who answers for its outcomes — to a customer, a regulator or a court.

So the first correction is to the framework: the model inventory must include third-party models, and "we cannot inspect it" is a risk to be managed rather than a reason for exclusion.

What to obtain from the vendor

Ask for what a model card would contain, and treat reluctance as information:

• Intended use and, critically, out-of-scope uses.
• Performance, disaggregated by relevant subgroup — aggregate accuracy can hide substantially worse performance on a minority group, which is the exposure that matters here.
• Training data provenance and population, so you can judge whether it resembles yours.
• Monitoring and retraining cadence, and how you are notified of a version change.
• Explanation capability — can it tell you why a particular customer scored as they did? Without this you cannot answer a complaint or an appeal.

Get the version-change notification into the contract. A silent model update changes your decision behaviour with no change on your side, and discovering that from a complaint is the worst way to learn it.

What to build regardless

Your own monitoring of outcomes, disaggregated. You may not see inside the model and you can see what it does: score distributions over time, and outcome rates by protected characteristic. Distribution shift and disparate impact are both detectable from the outside.

A decision log recording which vendor version scored each case, the score, the decision and the reason. This is what makes the process auditable and contestable even though the model is opaque.

A fallback. What the process does when the vendor is unavailable, and what it does if you have to stop using the model at short notice.

The question to put to the business

If this is a consequential decision about a person — credit, employment, insurance — an opaque third-party model may not be defensible under sector rules regardless of contractual comfort. That is worth establishing with legal before the integration is built, because the answer may be that a less accurate but explainable model is the only viable option.