At 01:50 an upstream system renames a column and the nightly customer model's join silently yields 4,000 rows instead of 1.2 million. Every test on that model is a not-null check on columns that survived, so the model passes. At 02:30 reverse ETL syncs it to the CRM. By 09:15 sales reports that lifecycle stage and account owner are blank on more than a million accounts and overnight campaigns fired against the wrong segment. Nothing errored. What failed, and which design decision made it possible?

The trigger

A renamed column turned a join predicate into one that almost never matches. The query is still valid SQL and still returns a correct-looking result set, just a 300-times smaller one. This is the hardest class of data defect to catch, because the output is well-formed and every value in it is right.

Why it propagated

The sync was configured in full-sync mode, whose contract is "make the destination match the query result". Absence from the result is therefore an instruction, and the instruction is clear the field. The same semantics as rsync --delete, applied to a system of record that 1,200 people use.

Two properties turned a bad model run into a business incident. The sync ran on its own schedule rather than on the model's success, so the model's failure state was information the sync never saw. And the sync had authority to clear fields the CRM's own users also write, so there is no clean restore: the previous values exist only in whatever snapshot the CRM keeps, and any manual edits made since the last sync are lost with them.

Why detection lagged

Everything was green. The orchestrator reported success, the tests reported success, and the reverse-ETL tool reported roughly 1.2 million records synced, which is exactly what a normal busy night looks like from its side. The only signal that would have fired is one nobody had: row count against the trailing median. Seven hours passed before a human noticed, and the notice came from sales rather than from the platform.

The structural fix versus the tempting local fix

The tempting fix is a not-null test on the renamed column. That catches this instance and not the class, which is "a query that is correct and returns far fewer rows".

The structural fixes, in the order they pay off:

1. A blast-radius guard on the sync. Refuse to run when the source row count deviates more than roughly 10% from the trailing 7-day median, and page instead. One configuration line, and it stops this failure and the next four like it.
2. Gate the sync on the model's published state, not on a clock. The sync consumes a contract with a freshness and completeness status; a failed or stale model means no sync.
3. Remove clear authority. Sync into fields the operational system treats as advisory and never writes itself, so the worst case is stale advice rather than erased records.
4. Volume and distribution tests on the model, asserting on row counts and on the distribution of the key columns, not only on nullability.

The general lesson

Reverse ETL makes a warehouse table into a production dependency while keeping a warehouse table's guarantees. The asymmetry is the whole risk: the analytics side treats a rerun as free, and the operational side treats a write as final.

When this is not a design error

For a small reference dataset where the warehouse genuinely is the only writer, such as a territory mapping or a product catalogue push, absence really should mean removal, and adding delta tracking is complexity for nothing. The test is ownership: if any human or system on the destination side can edit the field, the sync must never clear it.