You reprocess six months of events through an updated job to fix a calculation error. Support is flooded with customer complaints. What went wrong?

What the interviewer is testing

Whether you know that replay requires side-effect-free jobs, which is the constraint that makes Kappa architecture work or fail.

What went wrong

The job has side effects. Replaying six months re-executed them: six months of emails, push notifications, webhook calls to partners, or payment operations, compressed into a few hours.

The calculation fix was correct. The replay was not safe because the job was never designed to be replayed.

The design requirement

For a log-replay architecture to work, processing jobs must be deterministic and free of external side effects. Any side effect must be:

• Separated into a distinct job that is not replayed — the calculation job writes state, and a separate dispatcher acts on changes
• Or guarded by a replay flag in the job's context, so effects are suppressed during reprocessing — simple and effective, and it must be a first-class parameter rather than a comment
• Or idempotent with a deterministic key so the downstream deduplicates, which works for notifications with a stable id and does not work for anything with a time-based meaning

The general architecture that avoids the problem: reprocessing writes to a new output, which is compared and validated before consumers are switched to it. Nothing external is touched during reprocessing at all.

The recovery now

Stop the replay. Notify affected customers with an acknowledgement rather than waiting for complaints to accumulate. Contact partners who received duplicate webhooks, since their systems may have acted. Then complete the reprocessing into a separate output and cut over.

What a strong answer adds

The pre-replay checklist that should exist as a standing artifact: what side effects does this job have, are they suppressed, where does output go, who consumes it, what is the rollback, and has it been tested on a short window first. Reprocessing a day before reprocessing six months would have caught this at 1/180^th of the cost.

Common weak answers

Blaming the operator. Adding rate limiting to the replay, which slows the damage rather than preventing it.