Every infrastructure pull request passes the policy-as-code check. A quarterly scan finds 40 object storage buckets without the required encryption setting and 12 with public access. Nothing has been merged that would create them. Where is the gap?

The first three things I would look at, and why in that order

1. How the 52 resources were created. If they do not appear in any merged plan, they were made outside the pipeline — console, CLI, a vendor integration, or another automation with its own credentials. That single fact relocates the whole investigation.
2. What the policy evaluates. A check running against a plan evaluates intent at merge time. It says nothing about what exists now.
3. Whether anything reverts drift. If the pipeline only applies on change, a resource altered by hand stays altered until someone touches that module again, which may be never.

The diagnosis

The control sits at plan time and the finding is at runtime, and nothing connects them. Three distinct gaps usually contribute:

• Creation paths that bypass the pipeline. Standing write credentials on a cloud account are a path around every pipeline-based control, and they are almost always present for break-glass reasons.
• Manual mutation of pipeline-created resources. Someone disabled encryption during an incident at 02:00 and the plan has not run since. The code says one thing, the account says another, and no process compares them.
• Resources predating the policy. The check applies to changes; nothing sweeps what already existed when the rule was written.

The misleading signal

A green pipeline reads as compliance, and it is evidence of exactly one thing: no merged change violated the rule. Reporting it as a compliance metric is the substitution that lets the estate drift while the dashboard stays green — and it is the more dangerous error, because a red dashboard gets attention while a falsely green one earns trust.

The fix, in order

1. Add a runtime evaluation — continuous scanning of actual resource state against the same policies — and treat plan-time and runtime as two controls with two coverage numbers, not one.
2. Add admission control at the cloud API where the platform supports it, so a non-compliant resource cannot be created by any path, console included. This is the only placement that closes the bypass rather than reporting it.
3. Reconcile continuously, so hand-made changes are reverted or flagged within minutes rather than at the next quarterly scan.
4. Remove standing write credentials in favour of time-limited, logged elevation. Until this is done, every other control is advisory.

The alert that would have caught it earlier

Count of resources failing policy at runtime, reported daily, with the daily delta as the actionable signal. A quarterly scan finding 52 resources tells you nothing about when they appeared; a daily count would have shown 3 on the day someone changed a module's default.

When this is more than the risk deserves

For a small estate inside one account with two engineers, the quarterly scan is a reasonable control and building admission control is disproportionate. The threshold is roughly where the number of people who can create resources exceeds the number who know the policies — because beyond that point, drift is produced by ordinary competent work rather than by carelessness, and no amount of training reduces it.

Choose runtime scanning first and admission control second, because scanning is a read-only addition that can be deployed in days while admission control blocks creation and will break something in its first week. What admission control costs is a new failure mode: a policy service on the creation path means a policy outage becomes an inability to deploy, so it needs the availability engineering of a production dependency. Cloud providers have shipped organisation-level guardrails for this since about 2018, which makes the build-versus-configure question worth asking before anything is written.