Consumer lag has grown from 30 seconds to 4 minutes over a week. Throughput and error rates are normal. What is happening and how urgent is it?

What the interviewer is testing

Whether you recognise that steadily growing lag is a capacity emergency rather than a curiosity.

The diagnosis

Processing rate is below arrival rate, and has been all week. The system will not recover on its own — the gap compounds, and lag grows without bound until something breaks: retention expiry causing data loss, disk pressure on brokers, or the consumer falling so far behind that catching up is no longer feasible.

This is more urgent than it appears, because the metrics people watch all look healthy. Throughput is "normal" precisely because the consumer is saturated — it is processing at its maximum, which is why it cannot catch up.

The likely causes

Traffic growth that crossed the consumer's capacity a week ago.

A processing slowdown — a downstream dependency got slower, state grew large enough to affect lookups, or a deployment introduced a more expensive code path.

Partition skew, where one partition is hot and its single consumer is the bottleneck while others idle. Check per-partition lag, not the aggregate — this is the case where adding consumers changes nothing.

Rebalancing churn, where consumers are repeatedly reassigned and lose processing time.

The response

Check per-partition lag first, since it distinguishes a capacity problem from a skew problem and they have different fixes.

If capacity: scale consumers up to the partition count; beyond that, increase partitions. Confirm headroom is sufficient to drain the backlog, not merely to keep pace — catching up requires processing faster than arrival for a period.

If skew: address the key distribution, or split the hot key onto dedicated capacity.

What a strong answer adds

The alerting fix so this is caught on day one: alert on sustained growth in lag over a window, not on an absolute threshold. Absolute thresholds either fire on normal deployment spikes or miss slow growth entirely, which is exactly what happened here.

And measure lag in time rather than records, so it is comparable to the freshness objective.

Common weak answers

Treating four minutes as acceptable because it is small. Adding consumers without checking per-partition lag.