Match Confidence Band

An insurer consolidates three customer masters. Matching produces a score for every candidate pair. Above 0.95 the records are obviously the same person; below 0.60 they are obviously not. Between them sit the cases that decide whether the programme succeeds: the same name at two addresses, a married name, a company and its subsidiary sharing a registered address, twins.

The band is not a tuning detail. It is the design decision that sets both the error profile and the ongoing headcount, and it is usually discovered rather than chosen.

Why it matters

The two errors are not symmetric, so the threshold cannot be set to minimise total error. A missed match leaves a duplicate: visible, annoying, eventually reported by a user. A false merge combines two people, is nearly invisible — the record simply carries more history than it should — and can expose one customer's data to another, which is a disclosure rather than a data-quality issue.

That asymmetry means the threshold should be set to prefer duplicates over merges, which widens the band and increases the review load. A programme that optimises for automation rate is optimising for the wrong thing.

Implementation patterns

• Publish three numbers continuously: count above the merge threshold, count inside the band, and the reversal rate — how often a reviewed decision is later undone. The reversal rate is the one that tells you the thresholds are wrong.
• Staff the band permanently. Typically 70–85% of records match with high confidence; the remainder is not a migration backlog, it is a steady-state queue fed by every new record.
• Route by error cost, not by score alone. A pair involving a high-value account or a sensitive product deserves review at a score that would auto-merge elsewhere.
• Keep merges reversible. Store the source records and the decision, so an unmerge is an operation rather than an archaeology project.
• Re-run matching when rules change, and diff the outcome against the current state rather than applying blindly, because a rule improvement that flips 4,000 existing merges is an incident.

Industry example

The statistical basis for probabilistic record linkage goes back to Fellegi and Sunter's 1969 paper, and the pattern is identical across customer consolidation, supplier onboarding, patient record linkage and sanctions screening. The regulated versions make the asymmetry explicit: linkage standards in healthcare and financial crime screening both mandate human adjudication of the middle band rather than a single automated cut, precisely because the false-positive and false-negative costs differ in kind. The lesson generalises: wherever an automated decision can be wrong in two ways with different consequences, the band is where the process lives.

Failure scenarios

• A threshold tuned to maximise automation, producing a small number of false merges that surface years later as a privacy incident.
• The review queue assigned as a part-time responsibility, growing until someone clears it by approving in bulk — which converts a reviewed decision into an unreviewed one while preserving the audit trail's appearance.
• Irreversible merges, so a discovered false merge cannot be corrected without reconstructing state from backups.
• Matching rules improved and applied in one pass, silently restating thousands of records and every downstream aggregate built on them.
• No reversal-rate metric, so nobody learns that the thresholds drifted out of calibration after a data source changed.

Trade-offs

Narrowing the band raises automation and lowers cost, and moves errors from the queue into the data. Widening it does the reverse. There is no setting that avoids the choice, and the honest framing for a steering committee is a number: at this threshold we expect roughly this many duplicates and this many merges, and here is which of those we can live with.

When not to use it

When the parties can be matched by a reliable shared identifier — a national company number, a verified account identifier — probabilistic matching is the wrong tool entirely and the band should not exist. Use the identifier, and reserve fuzzy matching for records that lack one.

And when only reporting needs a unified view, the whole apparatus may be avoidable: publish a cross-reference table as a read-only service and let consumers join through it. That gives analysts one view without any system having to write a merged record, and a surprising number of consolidation programmes discover after two years that this was the entire requirement.

Interview question

Q: You are consolidating customer records across three systems. Where do you set the match threshold, what do you staff, and what would tell you six months later that you set it wrong?

What a strong answer covers: the asymmetry between a duplicate and a false merge, including the disclosure risk · setting the threshold to prefer duplicates · the 70–85% auto-match shape and the band as permanent staffing rather than a backlog · routing by error cost as well as score · reversibility of merges as a design requirement · the reversal rate as the calibration signal · re-running rules as a diff rather than a blind apply · and knowing when a reliable shared identifier makes the whole approach unnecessary.

Quick check

Quiz: Why should a match threshold be set to prefer duplicates over merges? Because a duplicate is visible and gets reported, while a false merge is invisible, carries a disclosure risk, and may be discovered years later.

Flashcard: Which metric shows the match thresholds have drifted? The reversal rate — how often a reviewed decision is later undone — not the automation rate or the queue depth.