Multimodal RAG Platform · View 06 of 16
Indexing & Embedding Pipeline
How a normalised chunk becomes a retrievable, governed index entry.
Copy
PNG
PDF
⋯
Editable source
SVG
draw.io
All views
Intake
Intake
Chunk Queue
at-least-once
Chunk Queue...
Dedup & Hash
content sha256
Dedup & Hash...
Enrich
Enrich
Context Header
Claude Haiku 4.5
Context Header...
PII Redaction
policy driven
PII Redaction...
Metadata Tagging
ACL, source, date
Metadata Tagging...
Embed
Embed
Text Embedding
1024-d, batched
Text Embedding...
Image Embedding
joint image-text
Image Embedding...
Sparse Terms
BM25 analysis
Sparse Terms...
Write
Write
Vector Index
tenant namespace
Vector Index...
Lexical Index
Lexical Index
Analytical Tables
typed rows
Analytical Tables...
Verify
Verify
Recall Probe
canary queries
Recall Probe...
Catalog & Lineage
version pinned
Catalog & Lineage...
below threshold, reindex
below threshold, reindex
Indexing & Embedding Pipeline
Indexing & Embedding Pipeline
Queue / topic
Queue / topic
Application we own
Application we own
Security / platform
Security / platform
Data store
Data store
Decision point
Decision point
failure / alternate
failure / alternate
Enrichment runs on the Batch API where latency allows, at half the per-token cost.
Enrichment runs on the Batch API where latency allows, at half the per-token cost.
v 1.0 · owner Solution Architecture
v 1.0 · owner Solution Architecture
Text is not SVG - cannot display
Enrichment
A short situating header is prepended to each chunk
Written by Claude Haiku 4.5, run on the Batch API at half cost
PII redaction applies before anything is embedded
Three index channels
Dense text vectors for semantic similarity
Image vectors for visual similarity
Sparse BM25 terms for identifiers and exact matches
Verification
Canary queries probe recall after every index build
Below threshold triggers reindexing, not a silent pass
Catalog pins the parser, chunker and embedding model versions
