A platform ships a major architectural change and declares it successful because it was delivered on time. What should have been measured?

The problem

Delivery is an output, not an outcome. Shipping on time says the project was managed well; it says nothing about whether the change achieved anything.

Architectural changes are especially vulnerable to this because their benefits are indirect — lower change cost, better reliability, improved scalability — and none of those is measured by a delivery date.

What should have been measured

Stated as a hypothesis before the work, with numbers:

If the goal was change cost: - Lead time for a typical change in the affected area, before and after. - Number of teams that must coordinate for a typical change. - Change failure rate and time to restore.

If the goal was reliability: - Incident frequency and duration in the affected area. - Error budget consumption. - The specific failure mode that motivated the change — did it stop occurring?

If the goal was scalability: - The constraint that was binding, and where it now binds. - Cost per unit of work at current and projected volume. - Load test to failure, before and after.

If the goal was cost: - Unit cost, not absolute cost, so growth does not confound it.

What makes this hard and worth doing anyway

The baseline must be captured before the work. Retrospective baselines are unreliable and contested. This is the step most often skipped and the one that determines whether the evaluation is possible at all.

Attribution is genuinely difficult. Many things change simultaneously. The mitigation is to state the hypothesis narrowly and measure the metric most directly affected, accepting that the evidence will be suggestive rather than conclusive.

The timeframe is long. Change-cost benefits appear over quarters. That argues for leading indicators — teams coordinating per change, deployment frequency in the affected area — rather than waiting a year.

Why it matters beyond this project

Architecture investment competes for funding against features, and it loses when its benefits are asserted rather than demonstrated. An organisation that measures architectural outcomes builds a track record that funds the next investment, and one that does not is arguing from first principles every time.

It also produces the more uncomfortable and more valuable finding: the changes that did not deliver, which is how architectural judgement actually improves.