Claim Provenance

When data arrives from many external parties on different schedules with different reliability, you do not have facts; you have claims. A supplier's inventory feed asserts a quantity as of a moment, from a source with a track record.

Storing it as a bare number discards the two attributes that determine whether it should be trusted, and the system then treats a nine-hour-old claim from an unreliable supplier identically to a two-minute-old one from a reliable one.

Why it matters

The failure it prevents is specific and costly: accepting an order against a number that was true this morning. With provenance, staleness policy is a visible decision; without it, the policy is "trust everything equally", chosen by omission.

Implementation patterns

• Every value carries as-of, source and confidence. Confidence is derived from the source's fulfilment history rather than from how reliably they send files.
• Policy expressed against those attributes: fresh and reliable → sell · fresh and unreliable → sell with safety stock withheld · stale → longer promised lead time or hidden · very stale or feed failing → stop selling and alert, rather than selling into an unknown.
• Distinguish "no update" from "zero". A feed that stops is not a supplier with nothing in stock, and conflating the two either hides good inventory or sells inventory that is gone. This is the single most common modelling error in aggregation platforms.
• Age claims out explicitly, with a per-source staleness threshold rather than a global one.
• Isolate sources from one another — per-supplier workers, queues and error handling — so one broken or slow feed cannot block ingestion for everybody.
• Feed fulfilment outcomes back into confidence, closing the loop so the model improves rather than ossifying.

Industry example

B2B commerce platforms such as Udaan and Moglix aggregate supply from very large numbers of sellers whose data arrives asynchronously, in different formats, at different frequencies and with different accuracy. The architectural insight is that a single availability boolean forces one policy across sources with wildly different reliability, which necessarily either loses sales on good suppliers or accepts unfulfillable orders on bad ones.

The same shape appears in financial-data aggregation over hundreds of bank APIs, in shipping aggregation over carriers, and in insurance platforms aggregating underwriter quotes — anywhere the platform's product is assembled from third-party assertions.

Failure scenarios

• A single boolean or bare quantity, so staleness is invisible to every consumer.
• Feed silence read as zero stock, hiding available inventory.
• Feed silence read as unchanged, selling inventory that has gone.
• One global staleness threshold, wrong for both the hourly supplier and the weekly one.
• A shared ingestion pipeline, so one slow supplier delays everyone.
• Confidence never updated from outcomes, so a supplier who stopped fulfilling keeps their reputation.

Trade-offs

Carrying provenance costs storage on every row and forces every consumer to decide what to do about staleness, which is more work than reading a number. Some consumers genuinely do not care and will find the metadata noise.

The mitigation is to compute a decision once, centrally — a sellable quantity derived from the raw claims plus policy — and let most consumers read that, while the raw claims remain available for the consumers and investigations that need them. That gives the simplicity of a single number without discarding the evidence.

Interview question

"A customer orders an item your platform showed as in stock, and the supplier cannot fulfil it. Tell me every place your data model should have warned you, and what you would change so the next one is caught."