Embeddings as Managed Features

Suppose a team upgrades the text encoder behind item_embedding_v, re-embeds new items as they arrive, and leaves the 200 million existing vectors alone until a backfill can be scheduled. Nothing errors. The ranking model downstream still receives 768 floats per item and still returns scores. For the next nine days it compares vectors from two different geometries, and click-through falls by an amount that takes a week to attribute. A scalar feature such as days_since_signup means the same thing whichever job computed it. An embedding is a coordinate in a space the encoder defines, and it is meaningless without that encoder.

That one fact changes how embeddings have to be managed as features. materialisation-and-the-online-store covers how any feature reaches a low-latency store, and operating-a-vector-store-in-production covers rebuilding an index after a model change. This concept is about the feature contract: identity, compatibility and the cost of change for every model that consumes the vector.

What the version has to pin

An embedding feature's identity is the tuple

Each element changes the vector. Preprocessing covers tokenisation, truncation length and which fields are concatenated. The input snapshot matters because re-embedding an item whose description changed yields a new vector under the same encoder. Post-processing covers normalisation, PCA or truncation of a Matryoshka embedding, whose nested prefixes let one model serve several dimensions (Kusupati et al., 2022, Matryoshka Representation Learning, arXiv:2205.13147). Truncation changes the dimension, not the space, so it gives no compatibility across encoder versions.

The practical rule is that the version is part of the feature name or key, never a column updated in place. Every consumer model records the embedding version it was trained on, and the serving path refuses to join a version the model was not trained with. That turns the silent geometry mix above into a loud failure. The same pairing of features to training versions is what offline-online-parity requires of any feature, enforced far more strictly here because nothing about a wrong vector looks wrong.

The cost of an encoder change

Take 200 million items at 768 dimensions in 32-bit floats. One version occupies \(2 \times 10^8 \times 768 \times 4\) bytes, about 614 GB before index overhead. While a new version is backfilled, both copies are stored and the online store holds both.

Compute depends on the encoder. At an assumed 2,000 items per second per GPU, the backfill takes \(2 \times 10^8 / 2000 = 10^5\) GPU-seconds, about 28 GPU-hours, which is cheap. The expensive part is downstream. Every model that consumes the embedding was trained against the old geometry, so each must be retrained, re-validated and redeployed in lockstep. With five consumers owned by four teams, the migration becomes a coordination project whose calendar time dwarfs the GPU bill. Historical training data compounds this: point-in-time training sets built from months of logged embeddings must be regenerated under the new encoder, and the inputs as they looked then must still exist to do it.

Compatibility instead of backfill

Two families of technique try to make the new encoder's vectors usable where the old ones were.

Backward-compatible training. Shen et al. train the new embedding model \(\phi_{\text{new}}\) with an extra "influence" loss that scores its embeddings through the old model's frozen classifier \(w_{\text{old}}\),

where \(T\) denotes training sets and \(\lambda\) weights the compatibility term. New vectors can then be compared directly with old ones, so retrieval galleries need no backfill (Shen et al., 2020, Towards Backward-Compatible Representation Learning, CVPR 2020, arXiv:2003.11942).

Learned alignment. Rather than constraining the new model, learn a map \(f\) from the new space to the old. The simplest is orthogonal Procrustes: with matched embeddings \(X_{\text{new}}, X_{\text{old}} \in \mathbb{R}^{n \times d}\) for the same \(n\) items,

Hu et al. generalise this for industrial embedding teams with many consumers: a learned aligner transforms the latest embedding into any historical version, so downstream models keep working while the embedding team iterates (Hu et al., 2022, Learning Backward Compatible Embeddings, KDD 2022, arXiv:2206.03040).

The two approaches disagree about where the cost should fall. Compatibility-constrained training ties the new model to the old geometry. Shen et al. report no accuracy sacrifice on their tasks, but the constraint plausibly grows harder to satisfy as architectures and objectives diverge, and repeated upgrades accumulate the old space's limitations. Alignment leaves the new model free and accepts an approximation error at every consumer.

When it breaks

A linear map cannot fix a different notion of similarity. If the new encoder was trained on a different objective, say semantic similarity instead of co-purchase, no rotation recovers the old neighbourhoods, and the aligned vectors are quietly worse than either native space.

Mixed versions look healthy. Cosine similarity between vectors from two encoders is a well-defined number, and embedding spaces are anisotropic, so cross-version similarities can be large and stable while meaning nothing. Version checks must be structural, not statistical.

Upstream refreshes are version changes too. Re-embedding changed item text under the same encoder shifts vectors that consumer models memorised, so refresh cadence belongs in the feature contract alongside the encoder version.

Nobody owns the deprecation. Keeping old versions alive for unmigrated consumers doubles storage and serving cost, and without a removal date, platforms accumulate several live versions of the same feature.