Control Design vs Operating Effectiveness
The distinction between a control being correctly designed to address a risk and it actually having worked consistently over a period.
Auditors test both, and engineering teams routinely prepare only for the first — which is why the second is where findings come from.
Design effectiveness asks whether the control, if operating as described, would mitigate the risk. It is assessed by inspecting the design: is peer review required before merge, does the policy check cover the right conditions, is the approval independent of the requester.
Operating effectiveness asks whether it actually did, for every instance across the period. This is tested by sampling: pull forty deployments from the last twelve months and show the evidence for each. A control that worked for eleven months and was bypassed in December is not effective.
The engineering consequence is that evidence must be generated automatically and retained, not reconstructed at audit time. Reconstructing who approved what six months ago from chat history is the difference between a routine audit and a quarter lost to it.
The design insight that follows: automated controls test far better than manual ones, because the population is complete and the evidence is inherent. A pipeline that cannot deploy without a passing policy check produces a stronger result than a documented procedure requiring a person to remember.