A regulated firm collects control evidence by hand each year for its certification: screenshots, exported spreadsheets, sampled tickets. It wants continuous controls monitoring instead. The certification cycle is mid-flight and the auditor has accepted the current approach. Give the sequence.
Show the full answer Hide the answer
The sequence, each step reversible
- Pick three controls, not thirty. Choose ones that are already machine-evaluable and boring: access recertification, change approval, backup completion. The first controls are chosen for demonstrability, not for risk, because the objective of step one is the auditor's confidence, not coverage.
- Run automated collection in parallel with the manual process, for a full cycle, changing nothing about what is submitted. This produces the thing that makes the rest possible: a period where both methods covered the same control and can be compared.
- Reconcile the two, and expect to find the automated result is worse. Continuous evaluation typically discovers exceptions that quarterly sampling missed — this is the system working, and it is also the moment the programme is most likely to be cancelled. Agree in advance with the control owners and with risk that a rise in detected exceptions is an expected outcome, or the first report will be read as a regression.
- Walk the auditor through the mechanism before relying on it. They assess the control over the evidence: how the collector authenticates, whether its output can be altered, what happens when it fails. A collector whose failure mode is silence produces evidence of nothing, and an auditor will find that.
- Submit automated evidence for those three controls in the next cycle, with the manual process retained as a fallback that is not exercised.
- Retire the manual process for those three only, then repeat. Do not run a big-bang conversion across the control set.
Where it can diverge, and how you would know
The dangerous divergence is scope, not accuracy. The manual process sampled a population somebody defined by hand — often including systems the automated collector does not know about. If the collector enumerates from a source that is itself incomplete, coverage falls while the evidence looks stronger. Reconcile populations before reconciling results, and carry coverage as a reported number alongside the exception count.
The point of no return
Retiring the manual process for a control while the certification depends on it. Before that, the fallback is a fortnight of work. After it, the fallback is reconstructing a year of evidence that was never collected.
The rollback at each stage
Steps 1 to 4 are additive and cost only effort. Step 5 is reversible within a cycle. Step 6 is not reversible within a cycle, which is why it applies to three controls and not thirty.
How long it really takes
Two certification cycles for the first tranche, and almost none of the elapsed time is engineering. It goes on the auditor relationship and the exception spike — convincing an organisation that finding more problems is the intended result of the investment.
When not to do this at all
A firm with 15 controls and one annual audit should not build a monitoring platform. The manual collection is perhaps two weeks a year, and the automation will cost more than that to build and considerably more to maintain.
It flips when evidence collection consumes more than a few weeks a year, when the same control is evidenced repeatedly for multiple frameworks, or when the gap between quarterly checks is itself the risk — a control failing on day 2 of a quarter and discovered on day 90. That last case is the only one where the argument is about risk rather than cost, and it is the one worth leading with.