Dominance Screen

Seven candidate message brokers, eight evaluation criteria, four weeks booked, and a meeting scheduled to agree the weights. Fifty-six scores will be produced and most will not matter, because a weighting argument can only change the ranking among options that each win on something.

A dominance screen runs first and asks one question of each candidate: is there another candidate that matches or beats it on every criterion? If so, no weighting can make it win, so it leaves the evaluation without anyone's priorities being stated. In a real shortlist that removes four or five of seven, cutting 60 to 70% of the scoring work before the contentious conversation begins.

Why it matters

Technology evaluations fail in two characteristic ways, and the screen addresses both. They stall on weights, because agreeing priorities is a political act and the matrix makes it look like an arithmetic one. And they produce false precision, where a 0.2-point margin across eight weighted criteria is presented as a finding when it sits inside the noise of the scoring.

The screen is judgement-free: it uses only the scores and its output is checkable by anyone. What survives is the frontier - two or three options that each win somewhere - and disagreement there is genuine disagreement about priorities, which belongs with whoever owns them.

Implementation patterns

• Score on a scale wide enough to separate candidates, typically 1 to 5. Pass-or-fail criteria hide differences and make everything look dominated.
• Apply hard constraints before the screen, as filters rather than criteria: a candidate that fails a residency requirement or a licence policy is out, not scored.
• Include operability, hiring and exit cost as criteria, or the screen eliminates on capability alone and keeps the option that is hardest to run.
• Publish the eliminations with their dominator: "B is out, dominated by D". This is what makes the screen defensible when an advocate asks why their favourite disappeared.
• Run it before the proof of concept to choose what to prototype, again after, because measured numbers reorder the frontier, and again whenever a criterion is added - a new criterion can un-dominate an eliminated option, and that is the moment to bring it back.

Industry example

The screen is the pragmatic front end to structured methods such as the Software Engineering Institute's Architecture Trade-off Analysis Method, published in 2000, whose real output is the set of trade-off points where quality attributes conflict. ATAM-style analysis is expensive per option, so running it across seven candidates is waste and running it across the two or three on the frontier is what the method is for. The same ordering appears in production evaluations of databases and brokers: eliminate on published facts, then spend the benchmark budget only where a real trade-off exists.

Failure scenarios

• Scores invented to protect a favourite. Someone marks their preferred option 4 on a criterion it plainly fails, and the screen cannot eliminate it. The defence is evidence per score, not a better formula.
• Criteria chosen after the candidates, so adding "native support for X" because one option has it makes the screen decorative.
• Dominance declared on incomparable evidence: one candidate benchmarked, another scored from a data sheet.
• The frontier treated as a tie, with two survivors averaged into a compromise that has the weaknesses of both.
• Everything survives. With eight loosely-scored criteria nothing dominates, which means the criteria are too numerous or too vague to discriminate.

Trade-offs

The screen gives up nuance for speed: it is insensitive to how much better a candidate is, so an option marginally worse on one criterion and vastly better on another is kept but not promoted. It is only as good as the scores. And it concentrates the argument onto the two or three hardest comparisons - which is the point, and it makes the remaining meeting harder, not easier.

When not to use it

With three candidates and four criteria the screen is ceremony: compare them against two concrete scenarios and decide. It is the wrong tool when a single criterion dominates - if a residency obligation or a licence restriction settles it, that is a filter and the rest of the evaluation is theatre. And where every candidate wins somewhere, the screen correctly returns everything and the real problem is a criteria list that has grown past the point of discriminating; cut it to the four that would change the decision.

Interview question

Q: You inherit a technology evaluation with seven candidates, eight weighted criteria and three weeks left. The team is arguing about weights. What do you do in the first two days?

What a strong answer covers: separating hard constraints into filters; running a dominance screen on the existing scores to eliminate without touching weights; demanding comparable evidence per score; publishing each elimination with its dominator; and spending the remaining time on the frontier via concrete scenarios rather than on weighting.

Quick check

Quiz: Why can a candidate that another matches or beats on every criterion be removed before weights are agreed? No non-negative weighting can make a dominated option win.

Flashcard: Seven candidates and eight criteria. What comes before weighting? — A dominance screen: eliminate every candidate another one matches or beats everywhere, typically 60 to 70% of the scoring work, with no agreement on priorities needed.