The business wants to deploy a model that ranks loan applications, with a credit officer making the final decision. What must the architecture provide, and what will you insist on before go-live?

Classify first, because it determines everything else

This decides access to credit for individuals. Under any consequence-based classification it is high risk: it materially influences a decision about a person's financial access, the harm from being wrong is significant, and the affected person is entitled to contest it. Sector regulation for credit decisions applies on top.

Classification is not paperwork — it determines the obligations, and therefore the build.

What the architecture must provide

Decision traceability. For any application, months later, the ability to reconstruct: which model version scored it, on which input features with which values, what score was returned, what the officer decided, and on what stated reason. That is an append-only decision log, retained for the regulatory period, queryable by application. It is the artifact a complaint, an appeal or a regulator will require, and it cannot be reconstructed afterwards.

Explanation. The officer needs to know why the model scored as it did, in terms they can evaluate — the features that drove it and in which direction. Without it, "human oversight" is a signature.

Model inventory registration, as a precondition of serving rather than a declaration. The serving platform should refuse an unregistered model version.

Fairness monitoring in production, disaggregated by protected characteristic, running continuously rather than at validation — because performance drifts with the applicant distribution.

Version pinning and rollback. A model update is a change with the same seriousness as a deployment, and the ability to revert to the previous version with the decisions attributable to each is a requirement.

What I would insist on before go-live

A chosen and documented fairness criterion. The principal criteria are provably incompatible where base rates differ, so someone accountable — with legal and credit policy input — must decide which one applies and why. Engineering cannot settle this, and leaving it unstated means it is settled by whatever the training happened to optimise.

Independent validation. Someone who did not build the model assesses it. This is the second line of defence and, in most credit contexts, an expectation rather than a nicety.

A model card, including disaggregated performance and explicit out-of-scope uses.

Evidence that the human review is real. This is the condition most often waved through and the one that most often fails. Automation bias means reviewers accept recommendations at very high rates, and a review that always agrees is not a control. Before go-live I would want: the officer seeing evidence rather than a conclusion, no throughput target that makes scrutiny impossible, a recorded reason on every decision, and a committed measurement of the override rate with a defined response if it approaches zero.

Adversarial and edge-case evaluation, including applicants unlike the training distribution.

The position to hold

The model is not the control. Input validation, the explanation surface, the decision log, the fairness monitor and the appeal route are what bound the harm when the model is wrong — and it will be wrong. A design that relies on model quality alone has no answer for the cases it gets wrong, and those are the cases that produce the complaint.