Semantic Chunking
Splitting documents along their meaning and structure rather than at fixed character counts, because retrieval quality is bounded by chunk quality.
Chunking is the least glamorous part of a RAG system and the one that most determines its ceiling. A fixed-size split at 512 characters will cut a table from its header, separate a contractual condition from its exception, and orphan a code block from the explanation that gives it meaning. No embedding model or reranker recovers information that was destroyed at ingestion.
Structure-aware splitting respects the document's own boundaries — sections, headings, list items, table rows, function definitions — and produces chunks that are individually coherent. Where a natural unit exceeds the target size it is split with overlap, so a concept spanning the boundary appears in both.
The technique that repays effort disproportionately is contextual enrichment: prepending each chunk with its document title, section path and a one-line description of what the document is. A chunk reading "the limit is 40 hours" is useless in isolation and retrievable when it carries "Employment Policy > Overtime > Weekly limits".
Two related decisions. Retrieve small for precision, then expand to the surrounding parent section before sending to the model, which gets accurate matching with sufficient context. And keep structured metadata on every chunk — source, date, permissions, document type — because filtered retrieval on metadata is frequently more effective than any improvement in embedding quality.