You must pick an embedding provider this week for a feature whose usage nobody can forecast - the corpus could settle at 200 thousand documents or at 40 million. Design the decision so that being wrong costs one sprint rather than one quarter.

Requirements that drive the structure

The uncertainty is a 200x range in corpus size, and the thing that is genuinely hard to undo is not the vendor. It is the stored artefact: vectors of a fixed dimension, written into an index, derived from text you may or may not still have. Design around that one fact and the vendor becomes a config value.

The design

• Keep the inputs, not just the outputs. Store the extracted text and the chunk boundaries that produced each vector, in your own storage. If the text is gone, changing provider means re-crawling or re-parsing, and that is what turns a sprint into a quarter.
• Stamp provenance on every row: provider, model identifier, dimension, chunking version. Without it you cannot tell which vectors are stale after a switch, and the only safe action is to rebuild everything.
• Make re-embedding a routine job, run before you need it. Run it once on 1% of the corpus in week one so the throughput is a measured number rather than a hope. At a thousand documents a second, 40 million documents is about 11 hours; at a hundred a second it is nearly five days, and that difference decides whether a switch is a weekend or a programme.
• One module owns the provider SDK, exposing two functions - embed a batch, report the dimension. Count the call sites and keep the number in the single digits.
• Buy time, not capacity. Monthly terms until usage is known. A twelve-month commitment signed in week one converts a reversible technical choice into an irreversible financial one, which is the most common way teams lose optionality while believing they are protecting it.

The hard part

Dimension is stickier than the vendor. An index built for 1536 dimensions does not accept 768, so a provider change that changes dimension forces a rebuild of the index even if every vector can be recomputed. Two honest options: accept the rebuild and keep it cheap by keeping the text, or pay for a dimension-reduction step and lose a little quality in exchange for a fixed target. Do not pretend an abstraction layer makes this free - that is the layer that fails in practice, because it abstracts the call and not the artefact.

What it costs

Storing the source text alongside the vectors roughly doubles the storage footprint for the retrieval system - at the scale in question, a rounding error against the embedding bill. Re-embedding costs the inference fee twice. Total premium for optionality here is single-digit percentage of the feature's run cost, which is the test for whether an option is worth buying: if keeping the choice open costs more than about a tenth of the thing it protects, stop paying for it and commit.

How it fails

The provenance column is added later, after the first switch is already needed. Then nobody can distinguish vectors by model, the rebuild is all-or-nothing, and the "one sprint" estimate becomes a quarter of coordinated backfill under live traffic.

When this is the wrong answer

Below roughly a million documents, design none of this. Re-embedding a small corpus is hours of compute and a day of work, so the choice is already reversible and the optionality machinery is pure overhead. Write the trigger instead - revisit when the corpus passes 5 million documents or the monthly bill passes a named figure - and spend the week on the product.