beginner 3 min answer

A platform's golden-path generator changed 31 times last year. The page telling engineers how to use it was last edited 14 months ago. In the last quarter 9 of 14 new services hit the same two errors in their first hour, each costing roughly 40 minutes plus a message in the platform support channel. What failed, and which design decision made it possible?

beginnergolden pathdocumentationdriftsupport load
Show the full answer Hide the answer

The trigger

Nothing broke. The generator kept working and kept changing - a renamed field here, a new required value there - and each change silently invalidated a sentence on a page nobody was looking at. The two recurring errors are simply the two places where the page and the generator now disagree.

Why it was possible

The documentation is a copy of the generator's behaviour, and every copy drifts from its source unless something forces them together. That is the design decision: the page and the code are two independent descriptions of one thing, maintained by different acts of attention, with 31 opportunities a year to diverge and zero mechanisms to reconverge.

Two things make the drift invisible:

  • The support channel absorbs the error. An engineer gets stuck, asks, is answered in four minutes, and no ticket exists. The cost is real - 9 incidents at about 40 minutes is six hours of new-starter time plus the platform's interruptions - and it never appears in any queue, so nothing is ever prioritised against it.
  • Nobody owns a page. The generator has an owner and tests. The page has neither, and documentation is the first thing cut when a change is urgent.

Why detection lagged

There was no signal on the first hour. The platform measures whether the generator ran, not whether the person reached a running service, so a 64% first-hour failure rate was indistinguishable from success in every dashboard the team had.

The structural fix versus the tempting local fix

The tempting fix is to update the page and add "update the docs" to the pull-request checklist. That restores accuracy for a few months and recreates the same gap, because a checklist is a reminder, not a mechanism.

The structural fix makes the two descriptions one:

  • Generate the reference from the generator's own input schema, so a renamed field renames itself in the docs.
  • Make the quickstart executable. A test that runs the documented commands end to end on every generator change turns a stale page into a failing build. This is the single highest-value item here, because it converts a silent drift into a loud one.
  • Count first-hour outcomes: services created, services reaching a first successful deploy, and the gap between them, with the errors grouped. Nine of fourteen is a number someone will act on.

When this is the wrong answer

If the generator changes twice a year and has three steps, a hand-written page is cheaper and perfectly honest, and an executable quickstart in CI costs more than it saves. The threshold is roughly change frequency against readership: once the thing changes monthly and more than one new service a month goes through it, the copy will be wrong more often than right.

Common weak answers

  • "Write better documentation." The page was correct when it was written. Accuracy is not a writing-quality problem, it is a drift problem.
  • "Add a doc review step to every pull request." It slows the generator's own change rate and still relies on a person noticing which sentence went stale.
  • "Point people at the support channel." The channel is what hid the defect for a year.