Invariant Register

Hugging Face acquired XetHub in August 2024 and by 23 May 2025 Xet-backed storage was the default for new users and organisations on the Hub, replacing per-file Git LFS deduplication with content-defined chunking at roughly 64 KB chunks grouped into 64 MB blocks in a content-addressed store. Git LFS stayed supported. The commands consumers type did not change.

That substitution was possible because the consumer-facing contract was a command, not a storage layout. But "the command still works" is only a safe claim if somebody has written down which observable properties were ever promised — and, more importantly, which were not.

An invariant register is that document: the enumerated set of properties a consumer may depend on, and the explicit list of properties you are reserving the right to change. The second list does more work than the first, because consumers depend on whatever they can observe unless told otherwise.

Why it matters

Every integration a consumer writes encodes assumptions. Some are the interface you intended; the rest are whatever they measured. Without a register, the two are indistinguishable at change time, so every internal improvement becomes a negotiation about whether it is breaking.

The register is also where the context diagram earns its keep. The edges from the system boundary out to external consumers are the only place in a diagram set where a promise is distinguishable from an implementation detail, and an edge annotated with its invariants is the difference between a picture and a contract.

Implementation patterns

• Enumerate the promises concretely: the command surface, the URL shape, the bytes a consumer ends up holding, the authentication method, a stated throughput floor.
• Enumerate the non-promises just as concretely: how many bytes cross the wire, the local cache layout, which client library gives the fast path, object keys and bucket paths, checksum semantics you compute internally.
• Version the register and date each entry, so a consumer arguing a regression can be answered from the document rather than from memory.
• Make the client requirement observable. If a fast path needs a newer client, give the consumer a way to tell which path they got — a response header, a CLI flag, a log line.
• Run both implementations at once wherever possible. New repositories on the new backend first, old ones untouched, is what buys a long window; the nine months between acquisition and default in the example above is that window.

Industry example

The Hugging Face Hub change is the documented case. Byte-level deduplication means a one-line edit to a multi-gigabyte checkpoint transfers chunks rather than the whole file, which is a consumer-visible improvement worth announcing. The Python fast path ships as a separate hf_xet package introduced with huggingface_hub 0.30.0, and Git LFS remains supported for compatibility.

The honest residue is silent fallback: a consumer on an older client gets the previous path and no error. So the sentence the announcement has to carry is not "we changed the storage" but "if your client is older than this version you get the previous path, and here is how to tell which one you got."

Failure scenarios

• Consumers integrate against the non-promises anyway — presigned object URLs, bucket paths, checksums they verify themselves — and discover it only when the implementation moves.
• Silent degradation instead of a clean break. The user reports "downloads feel slow", nobody can tell which path they took, and the support load never closes.
• The register lists promises and omits non-promises, which is the common half-measure; consumers then treat everything unlisted as stable.
• A single-backend cutover. With no coexistence the communication problem changes entirely: a date, a maintenance window, a rollback statement.

Trade-offs

Writing the non-promises down is the part that costs political capital, because it means telling a large customer that something they rely on was never guaranteed.

When not to use it

If your consumers are internal and few, a register is overhead — a conversation and a consumer list do the work. It is also the wrong instrument when the thing changing genuinely is in the promise set: no register makes a URL change non-breaking, and the answer there is a versioned endpoint with a dated deprecation window. And if a change is invisible to consumers and only improves your own storage bill, it belongs in a changelog line, not an announcement.

Interview question

Q: You are replacing the storage layer under an interface thousands of external consumers already use, and the commands will not change. What do you publish, when, and how do you avoid a year of "it feels slower" support tickets?

What a strong answer covers: a register separating promises from non-promises with the non-promises stated explicitly; coexistence of both backends with new consumers on the new path first; an observable signal telling the consumer which path they got; the client-version sentence in the announcement; naming what the consumer should re-measure; and the recognition that consumers integrating against storage layout rather than a command need a versioned endpoint instead.

Quick check

Quiz: Which half of an invariant register does the most work? — The non-promises, because consumers depend on anything they can observe unless they are explicitly told not to.

Flashcard: A storage change under a stable command produces no errors. What is the support problem and the fix? — Silent fallback to the slow path on older clients; make the path observable and state the minimum client version in the announcement.