A producer added a required field. New consumers work. A consumer replaying from three months ago fails. Why, and what is the policy fix?

What the interviewer is testing

Whether you understand that a topic contains history, which makes schema compatibility a bidirectional problem.

Why it fails

The consumer's code expects the new required field. Records written three months ago do not have it, so deserialisation or validation fails on historical data.

Adding a required field breaks backward compatibility — new code cannot read old data. It happened to work for new records because those have the field.

This is the failure mode that distinguishes streaming from request-response: an API change affects future calls only, while a topic change affects every consumer replaying from any retained offset, including a consumer that starts from the beginning next year.

The fix now

Make the field optional with a default. Consumers then read old records successfully, applying the default, and read new records with the real value. If the field is genuinely mandatory for business logic, the consumer handles its absence explicitly for the historical range rather than failing.

The policy fix

Full compatibility as the default mode for any topic with consumers you do not fully control — new code reads old data and old code reads new data.

Enforced by a schema registry at publish time, so an incompatible schema is rejected by the producer's own client before it reaches the topic. That moves the failure from production to development, which is the whole value.

Additive change only: add optional fields with defaults, never change a type, never repurpose a field's meaning.

What a strong answer adds

The change that no compatibility check detects: changing the semantic meaning of a field while keeping its name and type — currency changed, unit changed, a status code repurposed. Every automated check passes and every consumer is now wrong. For those, the answer is a new field or a new topic, and the only defence is review discipline plus documented semantics in the contract.

And sizing retention deliberately, since the compatibility window must cover the longest replay you intend to support.

Common weak answers

Telling the consumer not to replay that far back. Versioning the topic for every change, which multiplies operational surface.