A marketplace's search index lags several minutes behind catalogue updates and sellers complain their listings appear stale. Compare synchronous indexing, CDC pipelines and read-after-write patching at query time.

Understand who is complaining first

The seller who just edited a listing, and the buyer searching for it, have completely different requirements. The seller needs to see their own change immediately, or they will edit again and create duplicates. The buyer does not know the listing changed and cannot perceive a two-minute delay.

This asymmetry is the whole design. Solving global freshness to fix a seller's perception is enormously expensive and unnecessary.

The three approaches

Synchronous indexing — write to the database and the index in the same request. Buys: no lag. Costs: the search index becomes a hard dependency of the write path, so index unavailability means listings cannot be edited. Two systems without a distributed transaction means partial failures — a committed row with no index entry, or an index entry for a rolled-back row. Almost always the wrong choice, because it trades a large availability loss for a freshness gain most users cannot perceive.

CDC pipeline — the database's replication log drives an indexing consumer. Buys: the write path depends only on the database; ordering is guaranteed by the log; failed indexing is retried without losing the write; the index can be fully rebuilt by replaying. Costs: seconds to minutes of lag, and a pipeline to operate. This is the correct backbone.

Read-after-write patching at query time — the seller's recent edits are held in a short-lived store and merged into their result set. Buys: the seller's perception problem is solved directly, at the layer where it exists, with no change to the pipeline. Costs: complexity in the query path, and it works only for the actor who made the change.

The design that is actually right

CDC as the backbone, plus read-after-write patching for the actor who made the change. The pipeline stays simple and robust, and the only user who can detect the lag is given a consistent view.

Concretely: on write, record the change in a fast per-seller store with a TTL slightly longer than the p99 pipeline lag. When that seller searches or views their listings, overlay the recent changes onto the index results — replacing stale entries, adding not-yet-indexed ones, removing deletions.

What else to fix

Measure the lag as a first-class SLO — event time to index-visible time, at p99 rather than average, because the complaint is generated by the tail. Prioritise the pipeline: seller edits ahead of bulk imports and back-fills, so a large catalogue import does not delay one seller's price change behind a million rows. Expose the state in the UI — "your change is live, may take a moment to appear in search" converts a perceived bug into a communicated behaviour, which is the cheapest fix available and is usually skipped.