Default-Value Compatibility Rule

A producer adds one "optional" field to an event schema. The registry accepts the new version. Within minutes, consumers that have not been redeployed fail to deserialise every message on the topic.

The word doing the damage is "optional". In Avro, optionality is not a property a field has on its own. Schema resolution walks the reader's schema: for each field the reader expects, it looks for a value in the writer's data, and for each field the writer has that the reader does not, it skips. The asymmetry is the part people get wrong - a field present in the data but absent from the reader's schema is harmless; a field the reader expects and cannot find needs a default, or resolution fails.

A nullable union ["null", "string"] declares a field that may hold null. "default": null declares a field that may be absent. Only the second makes the schema evolvable, and the difference is one line.

Why it matters

Event schemas are the contract between teams that deploy independently, which is the whole reason a registry exists. A rule that depends on deployment order is not a contract, and this is the rule most estates get wrong without knowing: the registry's default compatibility mode checks one direction, and most organisations deploy in whichever order each team happens to choose.

The failure is also unusually total. It is not a degraded field or a missing value - the message cannot be read at all, so within 60 seconds of the producer's deploy every lagging consumer group stops dead on a topic that is otherwise healthy, while the producer that caused it sees nothing wrong on its side.

Implementation patterns

• Every added field gets a default. For a nullable field that means both the union and "default": null. Enforce it in a schema lint step rather than in review comments.
• Set the compatibility mode to match reality. BACKWARD means new consumers can read old data, so consumers deploy first. FORWARD means old consumers can read new data, so producers deploy first. FULL means order does not matter, which is the only honest setting for an estate that deploys independently.
• Prefer the transitive variants - FULL_TRANSITIVE checks against every earlier version rather than only the previous one, which is what you need because consumers are routinely several versions behind.
• Never remove a required field or change a type in place. Add the new field, migrate readers, then retire the old one - the same expand-and-contract discipline a database column needs.
• Test the direction the registry is not checking. A CI case that reads data written with the new schema using the previous schema costs a few lines and catches exactly this.

Industry example

Confluent's Schema Registry documents the compatibility modes and the rule that adding a field with a default is backward compatible while adding one without a default is not, and its default mode has been BACKWARD since it shipped in 2015. That default is sensible for the deployment order it assumes - consumers first - and is the wrong one for the independent deploys most platform teams actually have.

The same rule appears with different syntax across the ecosystem: Protocol Buffers gives every scalar field an implicit default and has no required fields in proto3 at all, which is the same lesson encoded into the language rather than left to the author. Where Avro asks for discipline, protobuf removed the option - a useful illustration of designing a format so the common mistake is unavailable.

Failure scenarios

• The "optional" nullable field with no default, breaking every un-redeployed consumer at once.
• A compatibility mode matching an order nobody follows, so the registry approves a change that stops production - correctly, by the policy it was given.
• Non-transitive checking, where a sequence of individually compatible changes is incompatible with the version a lagging consumer still holds.
• A removed field, which breaks in the opposite direction and therefore passes a FORWARD check.
• A schema id not in the registry, which looks like a compatibility failure and is actually a producer pointed at a different registry - worth ruling out before debugging the schema.

Trade-offs

Requiring defaults everywhere means you cannot express "this field must be present" in the schema. The validation has to move into the application, where it is easier to forget and harder to see. That is a genuine loss and it is the price of independent deployability: a schema that enforces presence is a schema that cannot be evolved without coordination.

FULL_TRANSITIVE costs the ability to make one-directional changes, so some legitimate simplifications become two-step migrations. It is worth paying once more than a handful of teams share a topic, and it is over-strict for a topic owned end to end by one team.

When not to use it

For a short-lived internal topic with one producer and one consumer that deploy together, the registry is documentation and the mode barely matters - the integration test covers it, and strict modes add friction without reducing risk.

And where presence genuinely must be enforced - a regulatory field that must never be absent - do not express that as a schema requirement on a shared event. Enforce it at the producer's boundary with an explicit validation, and keep the wire schema evolvable. Making the wire format the enforcement point means every future change needs a coordinated deploy, which is the cost the registry was bought to avoid.

Interview question

Q: A team adds an optional field to an Avro schema, the registry approves it, and every consumer that has not been redeployed starts failing. Explain why, and tell me what you would change in the schema, the registry configuration and CI.

What a strong answer covers: resolution driven by the reader's schema and the need for a default when an expected field is absent; the difference between a nullable union and a field with a null default; the compatibility modes, their directions and which deployment order each assumes; the transitive variants and why lagging consumers make them necessary; expand-and-contract for removals and type changes; and the CI case that tests the direction the registry does not.

Quick check

Quiz: Why is ["null", "string"] without a default still a breaking change? Because the union says the field may hold null, not that it may be absent - a reader that expects the field and does not find it in the data has no value to substitute, so resolution fails.

Flashcard: Which compatibility mode should an estate with independent deploys use, and why? FULL - ideally FULL_TRANSITIVE - because any weaker mode is a bet on a deployment order nobody is coordinating.