How would you evaluate an unfamiliar technology for a critical workload in one week?

What is being tested

Whether you prioritise evidence over reading, and whether you test the things that actually decide the outcome.

The week

Day 1 — define the question and the criteria. What specifically must this do that the incumbent cannot? What would make you reject it? Written down first, or the evaluation drifts toward whatever is interesting.

Days 2–3 — prototype against your actual workload. Realistic data volumes and realistic skew. Uniform synthetic data avoids exactly the hot-key problems that break production, so it produces a misleading pass.

Day 4 — test the failure modes. This finds more than any benchmark: kill a node under load, fill a disk, run a version upgrade, partition the network, exhaust a connection limit. How does it behave, how does it recover, and what does the diagnostic output look like?

Day 5 — the operational reality. Read the documentation for backup and tested restore, online schema change, monitoring, upgrades and troubleshooting. This is where you will actually live, and it is documented last in every product.

Throughout: find someone operating it at your scale and talk to them. Worth more than everything else combined, and usually obtainable through a user group or a network.

The source hierarchy

1. Your own measurement against your workload.
2. Someone operating it at your scale.
3. Operational documentation.
4. Published engineering accounts and incident reports, read critically.
5. Foundational papers, for understanding the mechanism.
6. Vendor material, which describes the happy path and the vendor's workload.

The critical-reading discipline

The most common research error is reasoning by analogy from a company whose constraints differ from yours. Ask of every published account: what scale, what team size, what constraint drove this, and does that constraint apply here?

Also search deliberately for migration-away accounts. People who left a technology are more informative than people who adopted it, and their write-ups are rarer and better.

What would make you reject it regardless of benchmarks

Nobody can operate it. Poor failure behaviour. Weak backup, upgrade and observability tooling. Uncertain governance or a single maintainer. Unacceptable exit cost for a one-way door. And a fit with your access patterns that is worse than the incumbent's.