A VP shows a dashboard placing your organisation in the Low performer cluster on the four delivery metrics and sets the goal "Elite by Q3". Review that goal. What is wrong with it what would you measure instead and what part of the request would you keep?

What the clusters actually are

The Low, Medium, High and Elite groups come from the DORA research programme's annual State of DevOps surveys, popularised by Accelerate in 2018. They are produced by cluster analysis over self-reported survey answers: respondents describe their own deployment frequency, lead time, change failure rate and restore time, and a statistical method groups similar answers together.

Three consequences follow, and they are the heart of the review:

• The clusters are descriptive, not a standard. The method has no notion of good; it separates response patterns. The behaviour that defines a group is whatever that year's respondents reported.
• The boundaries move, and so does the number of groups. The 2022 report produced three clusters with no Elite group at all, and the Elite group reappeared in 2023. A target defined by a label is therefore a target defined against a moving survey population.
• The framework itself changed. The 2021 report added reliability, or operational performance, as a fifth measure, because delivery speed and stability alone did not predict outcomes well enough. A goal phrased against the four-metric clusters is already a version behind.

Add the measurement problem on your own side: your dashboard numbers and the survey's numbers are not the same instrument. Yours come from pipeline and incident data with your definitions of "deploy", "failure" and "restored". The cluster boundaries come from people estimating. Comparing them is not a benchmark.

Why the target does not translate into work

"Elite by Q3" sets a target on an output of the delivery system and names no mechanism. The four metrics are diagnostics: deployment frequency and lead time describe throughput, change failure rate and time to restore describe stability. They tell you where to look. They do not tell you what to do, and a number can always be moved without touching the thing it measures.

That is where the gaming risk sits, and it is worth exactly one sentence in the review: deployment frequency rises if you split one deploy into four, and change failure rate falls if you narrow what counts as a failure.

What I would measure instead

1. Your own trend on your own definitions, published with the definitions attached.
2. The distribution, not the mean. p50 and p90 lead time. A mean hides the one change class that takes three weeks, and that class is usually the interesting one.
3. The decomposition of lead time, because that is where the work is. If lead time is 9 days and 7 of them sit between "ready for review" and "merged", then review latency is the constraint and a deployment-automation project cannot move the number at all. This is the single most useful number in the set, and the cluster label does not contain it.
4. One outcome metric alongside them - customer-visible error minutes or incident count - so that an improvement in the four numbers has to be corroborated by something the business feels.

What I would keep from the request

The instinct is right, and it should be said out loud rather than argued down. Measuring delivery consistently and making it visible is a genuine improvement, and the four metrics are a good set because they pair throughput with stability and resist one-sided optimisation better than most. Keep the dashboard, keep the definitions, drop the label.

A target I would accept: cut p90 lead time in half this quarter by attacking the largest component, and hold change failure rate flat while doing it.

Common weak answers

• "The metrics are vanity metrics." Overcorrection. The research finding that made them influential is that throughput and stability rise together rather than trading off, which is the opposite of the usual assumption and is worth defending.
• "Benchmark against the industry." There is no comparable instrument to benchmark against, and the survey population changes every year.
• "Add more metrics." The failure here is a target with no mechanism, and twelve metrics with no mechanism is worse.