A team's evaluation of a new datastore is a benchmark report: throughput at three concurrency levels, p99 latency, and a cost comparison. They want to commit. Review the evaluation. What is missing and what would you add?

What the evaluation actually establishes

That the system can do a synthetic workload fast on a clean cluster with no other tenants, no failures and no history. That is a necessary result and it is almost never the deciding one, because throughput is the property vendors optimise for and publish, which means it is the property least likely to differ from your expectations.

What is missing, in the order it will hurt you

1. Behaviour under partial failure. Kill a node mid-write. Partition two nodes from the rest. Fill a disk to 95%. The question is not whether it survives, but what it does to p99 and to correctness while it recovers — and how long the recovery takes at your data size, which is not the benchmark's data size.
2. Operational surface. Upgrades: in place, rolling, or restore-from-backup? Schema and topology changes under live traffic? What has to be hand-edited outside the declarative config — every one of those becomes undocumented state nobody can reconstruct later.
3. Restore, not backup. Time a restore of a realistic dataset, measured, once. A backup that has never been restored is a belief.
4. The failure history. Read the project's public incident reports and its issue tracker's oldest open bugs. What a system's maintainers have not fixed in three years tells you more than what it does in a benchmark.
5. The exit. What does moving off it cost, and is there a wire format or query dialect that makes that cost unbounded?

What to add, cheaply

Run a shadow of real production traffic for two weeks against a cluster sized as you would size it, with one deliberate failure injected per week. This costs a fortnight and tests all five of the above at once. Booking.com's practice of deciding product questions by running experiments rather than by argument is the same instinct applied to infrastructure: get evidence from the traffic you actually have.

The part that decides it

Add a line the benchmark cannot produce: who operates this at 3 a.m., and what do they do first? If the answer is "the person who ran the benchmark", the evaluation has measured the system and not the commitment.

When this is the wrong answer

For a reversible, contained choice — a cache in front of one service, a library, anything you can rip out in a sprint — the two-week shadow costs more than the mistake. Match evidence depth to reversal cost, not to how interesting the technology is. The heavyweight evaluation is for the choices that outlive the team that makes them: primary datastores, message buses, identity, anything that ends up in every service's dependency list.