advanced 2 min answer

A producer team wants to rename a field in a published event schema. What do you tell them and what makes the answer enforceable?

schema-registrycompatibilityavrolinkedingovernance
Show the full answer Hide the answer

What is being tested

Whether you know that a published event is an API with consumers you cannot enumerate, and whether your answer is a mechanism rather than a policy.

The answer

No, not as a rename. Do it as an additive change:

  1. Add the new field alongside the old one.
  2. Populate both.
  3. Migrate consumers to the new field over an agreed period.
  4. Deprecate the old field, and only remove it when usage is measured at zero.

A rename is, on the wire, a removal plus an addition. Consumers reading the old name get nothing — or worse, get the type's default value and cannot distinguish it from a legitimate one. That is silent wrong behaviour, not an error, and it is discovered by accounting rather than by monitoring.

What makes the answer enforceable rather than advisory

A schema registry with compatibility checks in CI. The registry stores every version of every schema and enforces a compatibility mode at registration time. A change that would break existing readers fails the pipeline. That is the entire value — mechanical enforcement rather than "we will be careful", which fails within the first year.

Choose the compatibility mode deliberately per subject:

  • Backward — the new schema can read old data. Required when consumers upgrade first.
  • Forward — the old schema can read new data. Required when producers upgrade first.
  • Full — both. Required when you cannot control the order, which in practice is most of the time.

Choosing the mode is choosing your deployment freedom, and it should be a conscious decision.

Why this becomes non-negotiable with a central log

LinkedIn's use of a central event log made schema governance an operational necessity rather than a preference. Once producers publish once and unknown consumers subscribe, the producer team cannot enumerate who would break. The freedom that makes the architecture valuable — adding a consumer without touching the producer — is exactly what removes the producer's ability to assess the impact of a change.

So the registry is not bureaucracy layered on top of the log; it is the control that makes the log safe to use.

The format point underneath

This is why the choice between JSON and a schema-carrying format matters more than payload size. JSON with no schema means the contract lives in documentation and in consumers' heads, and it breaks silently — a field's type changes from number to string and downstream code coerces it, producing wrong results rather than errors.

Protobuf and Avro carry explicit compatibility rules that a build can check. The decision that actually matters is schema governance, not serialisation efficiency.

What a strong answer adds

Asking why they want the rename. If it is clarity, a documentation or alias change may suffice. If the field's meaning is changing, that is a far more serious problem than a rename — consumers reading the same name would silently misinterpret it, and the correct move is a differently-named field with the old one deprecated.