Embeddings

Definition

An embedding maps content — text, images, audio — into a vector of numbers such that semantically similar items are near each other in the space. Similarity becomes a distance calculation.

What they enable architecturally

• Semantic search. Retrieval by meaning rather than by keyword, so "how do I cancel" finds a document titled "terminating your subscription".
• Retrieval for generation. Supplying a language model with relevant context, which is the dominant pattern for grounding model outputs in an organisation's own data.
• Recommendation and similarity, without hand-crafted features.
• Clustering and deduplication.

The architectural decisions that matter

Chunking. Documents must be split before embedding, and the chunk size is a genuine trade: small chunks retrieve precisely and lose context; large chunks preserve context and dilute the signal. Overlapping chunks and preserving structural context (the section heading, the document title) both help materially, and this is the parameter with the largest effect on retrieval quality.

Model choice and lock-in. Vectors from different models are not comparable. Changing embedding models means re-embedding the entire corpus, so the choice carries a real switching cost. Store the source content and the model version alongside the vectors so re-embedding is possible.

Storage. A vector index — either a dedicated store or an extension to an existing database. At modest scale the extension is usually correct; the operational cost of an additional store is real and the threshold for needing a specialised one is higher than it appears.

Freshness. Embeddings must be regenerated when content changes. That is an indexing pipeline with lag, and the lag must be monitored like any projection.

Hybrid retrieval. Pure vector search misses exact matches — a product code, an error identifier, a name. Combining keyword and vector retrieval, then re-ranking, consistently outperforms either alone, and omitting the keyword half is a common and avoidable weakness.

Failure scenarios

• Chunking chosen arbitrarily, which caps retrieval quality regardless of the model.
• Model changed without re-embedding, so old and new vectors are silently incomparable.
• No evaluation set, so retrieval quality is assessed by impression rather than measurement.
• Access control ignored — retrieval must respect the user's permissions, or the system becomes a data leak with a natural language interface.
• Vector search used alone where exact matching was required.

Interview question

"You are building retrieval over an organisation's documents. What are the three decisions that most affect quality?"