Deployment Order Coupling

Schema registries are usually explained as safety mechanisms: a compatibility mode is set, incompatible changes are rejected, breakage is prevented. That description is true and it hides the decision that actually matters.

A compatibility mode is a statement about deployment order. BACKWARD says a reader on the new schema can read data written with the old one, so consumers may be upgraded first. FORWARD says an old reader can read new data, so producers may go first. FULL says either order works. NONE says you are coordinating by hand.

Once seen this way, the usual questions answer themselves. Adding a field with no default is forward compatible and not backward compatible — so it is permitted exactly when you can upgrade every producer before any consumer adopts the new schema, and forbidden when you cannot.

Why it matters

It moves the decision from a registry setting to an organisational fact. "Can we control the order in which twelve teams deploy?" is answerable; "is this change safe?" is not, because safety depends entirely on the answer to the first question.

It also explains why FULL feels safe and costs so much. Choosing FULL is choosing to never control deployment order, and the price is permanent: only optional fields may ever be added or removed, for the lifetime of the topic. Teams adopt it for comfort and discover the constraint two years later, typically after 30 or 40 schema revisions, when a required field is genuinely needed.

Implementation patterns

• Set the mode per subject from the consumer population, not as an organisational default. A topic with one consumer owned by the producing team is not the same as one with 30 unknown readers.
• Write the reasoning next to the topic, so the next person knows whether the mode reflects a constraint or an accident.
• Use the transitive variants wherever history is replayed. BACKWARD_TRANSITIVE and FORWARD_TRANSITIVE check against every previous version, not just the last. A chain of individually compatible steps can be collectively incompatible, and a new consumer starting from offset zero reads version 1 — which on a retained event store can mean 10 billion records written under a schema nobody remembers.
• Give fields defaults by habit. A default turns most additions into changes permitted under any mode, which is why it is the standard advice and why it is also a modelling compromise — a default is a value you are asserting on behalf of data that did not have one.
• Treat a mode change as a migration, because relaxing FULL to FORWARD silently changes what future changes are allowed and nobody is notified.

Industry example

The mode names and semantics come from Confluent's Schema Registry, whose default is BACKWARD — which is why "add the field with a default" is the reflex answer in most Kafka estates, and why teams are surprised the first time they need the opposite direction. The same reasoning appears without a registry in protobuf's rules: reserving deleted field numbers exists because old and new binaries coexist in production and nobody can guarantee which deploys first.

Failure scenarios

• The correct change rejected, so a team adds a meaningless default to satisfy a mode chosen for unrelated reasons, and the default then appears in real data.
• Replay breakage: non-transitive modes pass every step, and a consumer reading from the beginning of the topic cannot decode the oldest records.
• Order assumed, not enforced: FORWARD is set, a consumer team deploys first, and decoding fails in production with nothing in the registry having objected.
• NONE chosen for velocity, after which the registry is a catalogue and the breakage simply moves to whoever deploys at the wrong moment.
• A mode changed quietly during a platform migration, widening what is permitted with no announcement.

Trade-offs

When not to use it

When producer and consumer are one deployable unit, the mode is irrelevant and setting a strict one adds friction with no protection — the compiler and the deploy already enforce the coupling. The same applies to a short-lived internal topic whose data is never replayed and whose retention is hours. Reserve the strict transitive modes for topics that are, in practice, an event store, because there the readers you are compatible with include readers that do not exist yet.

Interview question

Q: Your team needs to add a required field to an event schema consumed by twelve teams. The registry is on BACKWARD and rejects the change. Walk me through the options and tell me what you would actually do.

What a strong answer covers: that BACKWARD means consumers-first and the change is forward compatible rather than backward compatible; that switching the subject to FORWARD is legitimate only if producer-first deployment can be enforced across twelve teams; that adding a default is the usual pragmatic answer and is a modelling compromise, not a free move; and the transitive question — if this topic is ever replayed from the start, the non-transitive mode was already the wrong setting.

Quick check

Quiz: What does a compatibility mode actually constrain? The order in which the two sides of the interface may be upgraded — consumers first under BACKWARD, producers first under FORWARD, either order under FULL.

Flashcard: Why is adding a field with no default forward but not backward compatible? An old reader ignores an unknown field, so new data is readable by old code; a new reader encountering historical data has no value and no default for the field, so old data is not readable by new code.