A downstream consumer has been failing for four days. Your event log retains three days. What happens?

What the interviewer is testing

Whether you know that retention expiry is silent data loss and how it presents.

What happens

The consumer's offset points at data that no longer exists. Depending on configuration, it either fails to start, or — far more dangerously — resets to the earliest available offset and resumes, silently skipping a day of events.

That second behaviour is the default in many configurations, and it presents on a dashboard as the consumer recovering: lag drops to zero, processing resumes, everything looks healthy. The missing day is invisible until someone reconciles counts weeks later.

The immediate response

Establish what was lost — the offset range that expired, and therefore which entities were affected during that window.

Recover from the source if possible. If the events were derived from a database, the current state can be re-derived and the gap backfilled. If they were genuinely transient — clicks, telemetry — they are gone.

Reconcile the downstream state against the source of truth rather than assuming the consumer caught up.

The controls that prevent recurrence

Alert on consumer lag approaching the retention window, expressed in time rather than records. This is the alert that would have fired on day one and it is routinely missing.

Alert on offset reset, which is an unambiguous data-loss signal and is almost never monitored.

Set retention against the maximum tolerable consumer outage, deliberately. Three days means a consumer failing over a long weekend loses data, which is a decision nobody consciously made.

Configure the consumer to fail rather than reset on missing offsets, so the loss is loud.

What a strong answer adds

Sizing retention against the replay requirement rather than only against outages. If the architecture depends on reprocessing history to rebuild a derived store after a defect, retention must cover that — and tiered storage makes long retention affordable enough that this is no longer the constraint it once was.

And the reconciliation point: a derived store should be periodically compared against its source, because divergence from a missed window is otherwise undetectable.

Common weak answers

Increasing retention and moving on, without establishing what was lost. Assuming the consumer caught up because lag returned to zero.