Fitness Function Drift

A rule added in 2023 forbids the domain module from importing persistence. In 2026 the domain module imports it in fourteen places and the build is green. Nobody disabled the rule and nobody edited it. The package was renamed during a refactor, the matcher now selects zero classes, and an assertion over an empty set passes.

This is the failure mode automated governance introduces and manual review does not have. A review fails loudly by not happening; a check fails silently by happening vacuously. Ford, Parsons and Kua's Building Evolutionary Architectures (2017) defines a fitness function as a mechanism giving an objective assessment of an architectural characteristic. The assessment is only as good as the set examined, and nothing in a passing result reports the size of that set.

Why it matters

A control that is present but not operating is more dangerous than no control, because it consumes the attention that would otherwise go to the risk. Teams stop reviewing dependency direction by hand precisely because the rule exists, so drift accumulates for as long as the rule appears to work, and every available signal stays positive.

Knight Capital's 1 August 2012 loss is the expensive version of the same shape. A deployment reached seven of eight order-routing servers; the eighth still carried code retired in 2003, reachable through a flag the new code reused, and the firm lost more than $460 million in roughly 45 minutes. Nothing could report the difference between a safeguard being present and a safeguard being in effect.

The economics make drift the default: adding a violation to a baseline is one line under deadline pressure and removing it is a day. Repeat quarterly for three years and the baseline is the architecture.

Implementation patterns

• Assert coverage, not just compliance. The rule declares the minimum number of classes or modules it expects to examine and fails below it, so a rename breaks the build instead of silencing it.
• Keep a known violator: a fixture that permanently breaks the rule, verified in the same run. If the rule stops flagging it, the rule is broken. This is the only construction that tests the test.
• Expire every baselined violation. A recorded exception carries a date and fails the build when the date passes.
• Track the baseline as a metric: recorded-violation count per rule per build. A count that only rises is the drift.
• Keep mandatory rules in the pull-request gate. A check nobody sees within the hour is documentation.

Industry example

ArchUnit, the JVM architecture-testing library, ships FreezingArchRule for introducing a rule to a codebase that already violates it: the first run records existing violations in a violation store and later runs report only new ones, shrinking the store as violations are fixed. Its documented configuration includes freeze.store.default.allowStoreUpdate, which decides whether new violations may quietly join the store.

That feature is both the right answer to the adoption problem and the mechanism by which rules die. The store is a file in the repository that grows one line at a time, and each line is individually defensible. Those store flags are the actual control, and they are what most projects never revisit after the first green build.

Failure scenarios

• Rename drift. A package rename empties the selection; the assertion passes over nothing.
• Baseline creep. Recorded violations grow from 6 to 14 over three years, each addition approved.
• Stage relocation. The rule moves out of the pull-request gate for speed into a nightly job nobody watches.
• Scope split. A module is divided in two and the rule names only the original, so the new one is unguarded from birth.
• Tool upgrade semantics. A library version changes how a matcher treats generated classes and the effective scope shrinks with no code change.

Trade-offs

Meta-checking is not free. Coverage assertions add a number to maintain, so a legitimate module split produces a red build that is nobody's bug. Known-violator fixtures add a permanently non-compliant file that confuses newcomers, and expiry dates create scheduled failures.

The trade is worth making for rules whose violation is expensive to reverse — dependency direction, residency boundaries, where records of record may be written. It is not worth making for advisory rules about naming or layout, where the instrumentation costs more than the rule.

When not to use it

Do not instrument a rule the team has not agreed to; the machinery will be deleted along with the rule, framed as removing flaky tests. Do not add it to a codebase with three rules and one team either, where reading the rule file quarterly is cheaper. It starts paying when nobody can name every rule that exists.

Interview question

Q: Your organisation has 40 architecture rules across 12 repositories, all green. Convince me any of them enforce anything, and say what you would build to make the question answerable.

What a strong answer covers: greenness is not evidence. Measure each rule's selection size and recorded-violation count, then look for rules examining zero elements and stores that only grow. Build a per-rule coverage report published per build, plus known-violator fixtures for the rules that matter. Prioritise the expensive and irreversible rules, accept drift on advisory ones, and return every mandatory rule to the pull-request gate.

Quick check

Quiz: A rule has been green for three years and the codebase now violates it fourteen times. The two likeliest causes? The matcher selects zero elements after a rename, or the violations joined a frozen baseline one at a time.

Flashcard: The only construction that tests whether a fitness function still works? A permanently non-compliant fixture the rule must flag on every run, checked alongside the rule.