A retirement programme switches off System D after its 90 users are migrated. The disposition record says "no downstream consumers" - the integration catalogue listed D's outbound interfaces and there were none. Week 2 - a nightly pricing job begins producing output identical to the previous day; every value is plausible and nothing errors. Week 7 - a quarterly regulatory extract comes out 4% off and is queried. Month 5 - the discrepancy is traced to a currency-rounding reference table that a batch job inside D refreshed each night. What failed, and which design decision made it possible?
Show the full answer Hide the answer
The trigger
D was retired on evidence about interfaces, and the dependency was on data. Its nightly job wrote rows into a shared reference table that four other systems read directly. No interface existed because nobody built one: the producer and the consumers shared a database, so the coupling left no trace in any catalogue that models connections between systems.
The integration catalogue was not wrong. It was answering a different question.
Why it propagated
The table did not disappear - it stopped changing. Every consumer kept reading it, got well-formed values, and produced well-formed output. A stale read is indistinguishable from a correct read unless something asserts freshness, and nothing did.
Propagation was then arithmetic. Rounding reference data drifts slowly against reality, so the error accumulated: invisible in week 2, 4% by the quarterly extract, and present in every intermediate artefact in between. By month 5 the wrong values had been copied into the warehouse, into reports that had been approved, and into at least one external filing.
Why detection lagged
Three reasons, and they compound:
- Every monitor was an availability monitor. The job ran, exited zero, and wrote rows. There is no error to alert on in "the data is the same as yesterday".
- The first anomaly was absorbed. A pricing output identical to yesterday's looks like a quiet day. Equality to the previous run is the signal, and almost nobody alerts on it.
- The 4% variance was investigated as a reporting problem, because that is where it surfaced. It took four months to reach the retirement five months earlier, since no one connected a decommissioned application to a number in a regulatory extract.
The design decision that made it possible
The estate was modelled as applications and interfaces, with no attribute recording which system is authoritative for each data entity. The portfolio answered "what do we run and what talks to what" and could not answer "if this goes away, what stops being maintained". Those are different graphs, and only one of them was drawn.
A system of record attribute would have made the retirement gate trivially answerable: D is the authoritative producer for currency-rounding reference data, therefore retiring D requires a successor producer. The question was not hard. It was unasked, because the model had no place to put it.
The structural fix versus the tempting local fix
The tempting fix: add "check for shared database dependencies" to the retirement checklist. It is cheap and it fails, because it relies on someone remembering and on the departing owner knowing what reads their tables.
The structural fix, in order:
- Derive write dependencies mechanically, not by interview. Database grants, scheduled job inventories and query logs say who writes which table. Run that for every retirement candidate, and the finding is a list rather than a recollection.
- Give every reference data set a named producing system in the portfolio model, and make "producer retired with no successor" a hard block on disposition.
- Assert freshness where staleness is silent. A
last_refreshedcolumn with a job that fails when it exceeds the expected interval, and an alert on output identical to the previous run. This is the control that would have caught it in week 2 instead of month 5, and it is cheaper than everything else on this list. - Hold the retirement open for one full business cycle. Switching off is not the end of the programme; a quarterly process has not been tested until a quarter has passed.
The general lesson
Retirement risk lives in the data graph, not the application graph. An inventory that records systems and interfaces will approve this retirement every time. And the failure class to be most afraid of in rationalisation is not an outage - an outage is fast, loud and attributed. It is a system that keeps producing plausible output from data nobody is maintaining, because the cost compounds for months and the audit trail is contaminated before anyone asks a question.
When this is the wrong answer
When the retired system genuinely wrote nothing anyone else read, which is the common case for an end-user application with no batch jobs and no shared schema. Deriving the full write dependency graph for 140 retirements when 120 of them are front-ends over someone else's data is a tax on the programme. Gate on it only where the candidate owns a schema or runs a scheduled job - those two conditions select nearly all of the risk.