EA Metrics

Definition

Metrics for an enterprise architecture practice. The category is genuinely difficult, and the difficulty is why most practices measure artefacts instead.

What not to measure

• Artefacts produced. Documents, models and diagrams are output, not outcome, and optimising them produces exactly the shelfware the practice is criticised for.
• Reviews conducted. Measures activity, not effect.
• Compliance with standards, in isolation. High compliance with standards nobody should be following is not success.

What to measure instead

Portfolio health over time

• Applications in each lifecycle stage, and the trend. Is "contain" shrinking or growing?
• Applications retired — the metric nobody tracks and the one that indicates the practice can actually change things.
• Technology obsolescence exposure: components past or approaching end of support.
• Duplication: capabilities implemented by more than one system.

Delivery outcomes

• Lead time and deployment frequency, per team, over time.
• Teams on the paved road versus off it, and their comparative outcomes.
• Cross-team dependencies per delivery — a direct measure of whether boundaries are improving.

Decision quality

• Decisions recorded with alternatives and revisit conditions.
• Decisions actually revisited when their conditions triggered — which is the strongest signal that the practice is a learning one rather than a documenting one.
• Time from question to architectural decision.

Adoption

• Voluntary use of platform capabilities. The clearest signal that the practice is producing something people want.

The honest framing

Enterprise architecture's value is largely in decisions not made badly and problems not created, which is inherently hard to measure — you cannot count the duplicate systems that were not built.

The pragmatic response is to measure the conditions that produce good decisions (evidence available, decisions recorded, boundaries improving, standards followed voluntarily) alongside the portfolio and delivery trends that would eventually show a failing practice.

And to accept that a practice which cannot point to any changed outcome after a year is probably not producing value, whatever its artefact count.

Failure scenarios

• Artefact counts as the measure.
• Standards compliance without regard to whether the standards are right.
• No measurement at all, so the practice cannot defend its budget and is cut in the first downturn.
• Metrics that reward centralisation, so the practice optimises for its own control.

Interview question

"How would you demonstrate that an architecture practice is worth its cost?"