A grocery marketplace of the kind Instacart operates ingests retailer inventory feeds hourly and deduplicates on store id plus SKU. A retailer changes its SKU format and about 3% of distinct products now collide, so rows silently merge. Ingested row counts fall 3%, inside the 20% anomaly band. Schema tests pass and availability on the storefront quietly degrades. Which single check catches this?

The deciding property

Every other check examines the data you received; only reconciliation compares it against something outside your pipeline. A collision that merges two real products produces data that is internally consistent in every way: unique keys, valid types, no nulls, plausible counts, on time. There is no evidence of the loss inside the artefact, because the evidence was the rows that no longer exist.

A control total — "this feed contains 48,312 distinct products" — comes from the producer's own count before transport. The difference between that and your post-dedup distinct count is the loss, stated in products, with no inference required.

Why the other options fail

• The null-rate test on SKU. A reformatted SKU is populated and well-formed. Nulls would catch a missing field, which is a different and much louder failure. This test passes and reassures you.
• The uniqueness test on the output key. It passes by construction. The deduplication step made those keys unique — that is precisely the operation that destroyed the rows. A uniqueness assertion downstream of a dedup can only ever confirm that the dedup ran, which is the trap: the test most teams would write is structurally incapable of failing here.
• Tightening the row-count band to 5%. The strongest-looking distractor, and it does cross the 3% threshold. Two problems. Volume varies legitimately: a retailer onboarding 2,000 new lines, a seasonal range change or one store's feed arriving late all move counts by more than 5%, so at hourly granularity across hundreds of retailers this produces alerts most days and is muted within a fortnight. And it tells you that the count moved, never which products vanished, so every alert starts a one-hour investigation. It is a reasonable second line of defence and a poor first one.
• The freshness check. The feed arrived on time. Freshness catches a stalled producer, which is the failure teams instrument first because it is easy, and it has nothing to say about content.

What it costs

The producer has to send the total, which is a negotiation rather than an engineering task — and it is exactly the negotiation a published data contract exists to settle. Where a retailer will not send one, you can derive a weaker proxy: count distinct SKUs in the raw landed file before deduplication and compare that with the post-dedup count. That catches collisions introduced by your own pipeline, which is this case, but not loss that happened before the file was written.

Running cost is trivial: one count per feed, compared to a number already in the payload.

When this is the wrong answer

When you own both ends, reconcile against the upstream table instead of inventing a control total. An internal pipeline can compare its output's distinct key count against a query on the source, which is cheaper, needs no agreement and is exactly equivalent. Control totals matter when the data crosses an organisational boundary and the producer is the only party who knows what they sent.

Also skip it where the cost of the error is low. A marketing enrichment table losing 3% of rows is worth an anomaly band and nothing more. This check is justified here because the merged products become unavailable to shoppers, so a data defect is lost revenue in the same hour.