A content platform's search index lags the source by up to 30 minutes. Product wants it under 10 seconds. What do you assess?

What the interviewer is testing

Whether you interrogate the requirement and understand what indexing latency is actually made of.

First, decompose the 30 minutes

The lag is not one thing, and the components have very different costs to fix:

Capture latency — how long before a change leaves the source. Batch extraction on a 30-minute schedule is the answer here, and change data capture from the transaction log reduces it to sub-second.

Processing latency — enrichment, joins with data from other services, transformation.

Index commit latency — most search engines batch document writes and refresh the searchable view on an interval, because committing per document is expensive.

Replication to search replicas serving queries.

If the dominant term is a 30-minute batch schedule, the fix is straightforward. If it is distributed across all four, the work is larger.

The questions for product

Which changes need to be fast? A new listing appearing is usually urgent. A description edit usually is not. Differentiated freshness by change type frequently satisfies the real requirement at a fraction of the cost, and it is the question that most often resolves this.

What is the actual harm at 30 minutes? Sometimes the answer is a genuine business impact; sometimes it is a general preference for freshness.

Is 10 seconds needed, or is 60 acceptable? The cost curve is steep between them.

The design if it is genuinely needed

CDC from the transaction log rather than scheduled extraction. Streaming enrichment with denormalised data pre-joined at write time rather than looked up per document. Frequent index refresh, accepting the resource cost. And possibly a two-tier read: query the index, and overlay very recent changes from a small hot store, so freshness does not depend on index commit latency at all.

What a strong answer adds

The costs incurred: more expensive infrastructure running continuously, an operationally harder pipeline, and a rebuild path that must be preserved — the index will need rebuilding after a mapping change or a defect, and a streaming pipeline makes that harder to reason about than a batch one.

And reconciliation, because a streaming index diverges silently and search results that look plausible are the worst kind of wrong.

Common weak answers

Building the streaming pipeline without asking which changes need it. Increasing the batch frequency, which reduces the lag linearly and hits a floor.