An architecture practice reports 92% standards compliance (up from 61%), an application inventory at 100% coverage, and 40 reference architectures published. Over the same period, lead time for changes has risen from 9 to 17 days and the change failure rate is flat. What is happening, and what would you measure instead?
Show the full answer Hide the answer
The first three things I would look at, and why in that order
- The distribution of lead time, not the mean. A rise from 9 to 17 days is either everything getting slower or a subset getting much slower. If the median moved a little and the 90th percentile doubled, the cause is a gate that some changes hit and most do not, which points straight at the new compliance regime.
- Where the waiting happens. Split lead time into coding, review, waiting for approval, and waiting for another team. An EA practice can only plausibly affect the last two, and if the growth is in coding time, the architecture function is not the cause and the report is a coincidence.
- What "compliance" is measured on. Self-assessment, a checklist at project start, or an automated check against running systems? Self-reported compliance rising to 92% under management attention is the least surprising event in this data.
The diagnosis
Almost always: the practice is measuring its outputs, and the outputs are produced by adding steps. Compliance rose because a gate was added. Inventory coverage reached 100% because teams were required to file records. Forty reference architectures exist because producing them was the work. Each is a real activity with a real cost, paid in the lead time.
The flat change failure rate is the decisive signal. If the added governance were preventing defects, the failure rate would fall. It did not move, so the organisation is paying for the controls and receiving nothing measurable in return — the controls are checking properties that were not causing the failures.
The misleading signal
The compliance number itself, because it moved most and looks like progress. It is measuring adherence to rules whose connection to outcomes has never been demonstrated, and "92%" gives it a precision that hides the absent causal argument. The question that dissolves it: which incident in the last year would this rule have prevented? For most rules in most standards documents, there is no answer.
What to measure instead
- Lead time and change failure rate, owned by the architecture practice as its own scorecard rather than delivery's. If governance is working, failure rate falls without lead time rising.
- Boundary crossings per feature: how many independently owned services or teams must change to deliver a median piece of work. This is the closest thing to a direct measure of whether architectural boundaries fit the work, and almost nobody instruments it.
- Time-to-first-deploy for a new service, which measures the paved road rather than the rules.
- Proportion of decisions reversed within a year, which says whether decisions are being made at the right altitude with the right information.
- Unplanned cross-team work, the escapee category: work that arrived because someone else's design forced it.
The alert that would have caught it earlier
A standing comparison of the practice's own metrics against delivery's, published together. The failure here is not that the wrong things were measured; it is that they were measured in isolation, so a rise in one and a rise in the other never met on the same page. Any governance metric reported without a delivery metric beside it should be treated as incomplete.
When this diagnosis is wrong
If lead time rose because the organisation absorbed an acquisition, entered a regulated market, or grew from 30 to 90 teams in the period, the coordination cost is structural and the architecture practice may be the thing keeping it from being worse. The control is the counterfactual: find a part of the estate where the new gates do not apply and compare. If lead time rose there too, the governance is not the cause, and the right response is to say so rather than to defend the metrics.