DORA Performance Clusters

A VP shows a dashboard placing the organisation in the Low performer group on the four delivery metrics and sets the goal: Elite by Q3. The numbers on the dashboard come from the deployment pipeline and the incident tracker. The boundary they are being compared against does not.

The Low, Medium, High and Elite groups come from the DORA research programme's annual State of DevOps surveys, popularised by Accelerate in 2018. They are produced by cluster analysis over self-reported survey answers: respondents estimate their own deployment frequency, lead time, change failure rate and time to restore, and a statistical method groups similar answer patterns together.

That makes the clusters a description of a survey population in a given year, not a specification. Treating one as a target is a measurement-validity error before it is a management error.

Why it matters

Three properties of the method decide how the output can be used.

• The groups are descriptive. The clustering has no notion of good; it separates response patterns, and the behaviour that characterises a group is whatever that year's respondents reported.
• The boundaries move, and so does the number of groups. The 2022 report produced three clusters with no Elite group at all, and the Elite group reappeared the following year - so a cross-year "we moved up a tier" claim can be an artefact of the sample.
• The framework itself changed. The 2021 report added reliability, or operational performance, as a fifth measure, because delivery speed and stability alone did not predict organisational outcomes well enough.

What survives all of that is the finding that made the research influential and that most teams still get wrong: throughput and stability improve together rather than trading off. That is a claim about mechanism, and it is usable. A label is not.

Implementation patterns

• Publish your definitions next to your numbers. What counts as a deploy, as a change failure, as restored. Two teams with different definitions cannot be compared, and neither can you year on year.
• Report distributions, not means. p50 and p90 lead time. A mean hides the change class that takes three weeks, which is the one worth investigating.
• Decompose lead time by stage - commit to ready-for-review, review wait, review rounds, merge queue, release wait. If 7 of 9 days sit in review wait, a deployment-automation project cannot move the number.
• Pair the four with an outcome measure - customer-visible error minutes, incident count, or the fifth measure's reliability-target attainment - so improvements have to be corroborated by something users feel.
• Target the constraint, not the metric. "Halve p90 lead time by attacking review wait, holding change failure rate flat" is a goal with a mechanism.
• Re-read the current year's report before quoting a threshold, because last year's numbers are not this year's.

Industry example

The research itself is the example, and it is unusually transparent about method: survey instrument, cluster analysis, annual re-derivation, published reports since 2014 with the four metrics formalised in Accelerate (2018) and reliability added in 2021. The 2022 edition's three-cluster result, followed by the Elite group's return the next year, is the most useful single fact about the series: the groups are an output of each year's data rather than a fixed ladder.

Secondary commentary has also flagged the self-report problem directly: respondents who rate their own reliability highly are the same people choosing to answer a survey about delivery performance.

Failure scenarios

• A cluster label adopted as an OKR, which sets a target on an output with no named mechanism and no owner of the constraint.
• Deployment frequency gamed by splitting one release into four, improving the number and nothing else.
• Change failure rate gamed by narrowing what counts as a failure, which also destroys the series' usefulness as a trend.
• Cross-year comparison against moving boundaries, producing a reported improvement nobody can feel.
• The fifth measure ignored, so a team optimises throughput while reliability degrades and the delivery dashboard stays green.
• Benchmarking instrumented data against survey estimates, which is the error that starts all of the above.

Trade-offs

Using the clusters as a conversation starter is cheap and genuinely useful: they give a shared vocabulary, and the pairing of throughput with stability stops one-sided optimisation. Using them as a target buys an easily communicated goal and pays for it with gaming, cross-year noise and attention spent on a label rather than on the constraint.

The four metrics themselves are worth the instrumentation. They are few, they are hard to improve by accident, and they resist gaming better than most delivery measures - as long as the definitions are stated and the goal names a mechanism.

When not to use it

For individual or team performance management. The research measures organisational delivery capability, and applied to a person it becomes a quota.

For comparing two teams with different change profiles. A team shipping a regulated payment path and a team shipping a marketing site will differ on every metric for reasons that have nothing to do with capability.

As an acceptance criterion in a contract or an audit. The numbers are not defined precisely enough to be contractual, and the boundaries change annually.

When you have no instrumented baseline. Measure your own four numbers for a quarter first; a benchmark against an external population is worthless without them.

Interview question

Q: An executive wants "Elite performer status by Q3" on the delivery metrics. You think the goal is poorly formed but the instinct behind it is right. How do you redirect it without dismissing it, and what do you commit to instead?

What a strong answer covers: explaining that the clusters come from survey-based cluster analysis and that the boundaries and even the number of groups move between years - the 2022 three-cluster result is the concrete example · that the four metrics are diagnostics rather than targets · what to commit to instead: stated definitions, p50 and p90 lead time, the stage decomposition that names the constraint, and one corroborating outcome measure · the gaming risks named in advance, so the executive hears them from you first · and keeping what was right about the request, which is that delivery should be measured consistently and made visible.

Quick check

Quiz: Why can a team not benchmark its instrumented pipeline data against the Elite cluster boundary? — Because the boundary is derived from cluster analysis of respondents' self-reported estimates in a given survey year, so it is a different instrument on a moving population - and the number of clusters itself has changed between reports.

Flashcard: What fifth measure did the 2021 DORA report add, and why? — Reliability, or operational performance, because delivery throughput and stability alone did not predict organisational outcomes well enough.