# Architecture Decision Records — Multimodal RAG Platform Each record states the decision, what was rejected, and what would make us revisit it. --- ## ADR-001 — Normalise every modality into one chunk record **Status** Accepted **Context** Six input modalities with nothing structurally in common. Retrieval, ranking, citation and access control each need a uniform object to operate on. **Decision** Every extraction path emits the same record: a text projection, a modality tag, a provenance pointer and an optional parent reference. Modality-specific detail lives in the `asset` entity, not in the retrieval path. **Rejected** Per-modality indexes with a federated ranker. Rejected because ranking across independently-scored indexes requires score calibration that drifts, and because citation and ACL logic would be implemented six times. **Consequences** Retrieval, reranking and generation are modality-agnostic. The cost is that each new modality must be reduced to text well enough to be findable — which is a real constraint on, for example, purely visual content with no OCR-able text. **Revisit if** a modality arrives that genuinely cannot be projected to text without losing the thing users search for. --- ## ADR-002 — Index images as generated descriptions plus a native image vector **Status** Accepted **Context** Images carry meaning that OCR alone misses — chart trends, diagram topology, photographs. A joint image-text embedding retrieves visually similar images but poorly captures "what does this chart say". **Decision** Two channels. A vision model writes a description; that description plus any OCR text is embedded as text and becomes the primary retrieval channel. A joint image-text embedding is stored as a secondary channel for visual-similarity queries. The channels are fused at rerank. **Rejected** Image embeddings only — fails the dominant question type. Descriptions only — fails "find the diagram that looks like this". **Consequences** Indexing cost per image is higher and includes a model call. Description quality is now permanent index quality: a bad description makes an image permanently unretrievable, so this call uses the strong model, not the cheap one. **Revisit if** joint embedding models close the semantic gap enough to make descriptions redundant. --- ## ADR-003 — Store tables twice: linearised for retrieval, typed for SQL **Status** Accepted **Context** Users ask both "what does the pricing table say about tier 3" (retrieval) and "what is the total across all regions" (aggregation). The second has no answer in any chunk. **Decision** Tables are linearised to markdown and indexed as ordinary chunks, and separately loaded as typed rows into an analytical store. Numeric and aggregate questions are routed to generated SQL against those rows. **Rejected** Retrieval-only over linearised tables. Rejected because it produces confident arithmetic that is wrong — the worst possible failure mode for a system whose value proposition is groundedness. **Consequences** A routing decision is now required at query time, and generated SQL must be sandboxed and validated. Duplicate storage of table content is accepted. **Revisit if** table sizes make row-level loading impractical at corpus scale. --- ## ADR-004 — Hybrid retrieval with reciprocal rank fusion and cross-encoder reranking **Status** Accepted **Context** Enterprise queries contain identifiers — part numbers, error codes, policy references — where dense retrieval reliably fails. Pure lexical retrieval fails on paraphrase. **Decision** Dense and sparse retrieval run concurrently, each returning 50 candidates. Results merge by reciprocal rank fusion, then a cross-encoder reranks to the top 8. **Rejected** Weighted score fusion — requires calibration between incomparable score distributions and drifts as either index changes. RRF operates on ranks and needs none. **Consequences** Two indexes to maintain and keep consistent. A GPU reranker on the latency-critical path, budgeted at 400 ms for search plus rerank. **Revisit if** measured recall from a single channel matches the hybrid within tolerance on the golden set. --- ## ADR-005 — Prepend generated context to each chunk at index time **Status** Accepted **Context** An isolated chunk often lacks the terms a user would search by. "The rate increased to 4.2%" does not mention which policy or which year. **Decision** At index time, generate one short line situating the chunk in its document and prepend it before embedding. **Rejected** Larger chunks — improves context at a direct cost to retrieval precision. Query-time expansion only — does not fix a chunk that is unfindable in the first place. **Consequences** One model call per chunk at index time. Mitigated by using Claude Haiku 4.5 via the Batch API at half rate. Reindexing is required if the enrichment prompt changes, which is why the derived zone exists. --- ## ADR-006 — Small-to-big retrieval via parent references **Status** Accepted **Context** Precision favours small chunks; generation quality favours large context. **Decision** Embed small chunks; on a hit, return the parent section to the model. **Consequences** `chunk.parent_chunk_id` is load-bearing. Deduplication is needed when several child hits share a parent. --- ## ADR-007 — Enforce grounding before an answer is released **Status** Accepted **Context** The requirement is grounded, cited answers. Generation with retrieved context raises groundedness but does not guarantee it. **Decision** A verification step checks each claim against a retrieved span. Failure triggers one retry with a rewritten query; a second failure returns an explicit abstention. **Rejected** Trusting the generation prompt. Rejected because an unenforced instruction is not a control, and because groundedness then cannot be measured as a release gate. **Consequences** Added latency and an extra model call on the read path. Abstention rate becomes a tracked quality metric — a rising rate means retrieval degraded, which is exactly the signal wanted. --- ## ADR-008 — All model calls through a single gateway **Status** Accepted **Context** Model choice, cost control, redaction, caching, fallback and retry policy would otherwise be reimplemented in every service that calls a model. **Decision** One gateway owns routing by task, semantic caching, per-tenant budget caps, payload redaction, provider fallback and retry. No service holds a provider credential. **Consequences** The gateway is on the critical path and must be highly available. In exchange, a model swap, a price change or a redaction rule is a single-place change, and egress is auditable at one point. --- ## ADR-009 — Separate the asynchronous write path from the synchronous read path **Status** Accepted **Context** Ingestion is bursty and minutes-scale; querying is steady and seconds-scale. **Decision** Two independently scaled domains sharing only storage. Ingestion is queue-driven with idempotent, resumable workers; querying is a synchronous request path. **Consequences** Eventual consistency between upload and queryability, bounded at 15 minutes and stated as an NFR. Users are shown ingestion status rather than left to guess. --- ## ADR-010 — Enforce tenant isolation inside the retriever **Status** Accepted **Context** Multi-tenant corpora in a shared index. A filter supplied by the caller is a filter an attacker can omit. **Decision** The retriever derives tenant and ACL filters from verified token claims and applies them itself. Vector namespaces are partitioned per tenant; metadata uses row-level security. **Consequences** The retriever cannot be called in an unfiltered mode, including by internal batch jobs, which must present a service identity scoped to a tenant. Cross-tenant leakage is asserted in automated tests as a release gate. --- ## ADR-011 — Treat prompts, chunking configuration and eval sets as versioned code **Status** Accepted **Context** A prompt edit or a chunk-size change moves answer quality as much as a code change, but conventionally bypasses every release control. **Decision** These artefacts live in the repository, are versioned with the code, and pass the same CI gates — including the RAG eval gate and the cost gate. **Consequences** No hot-editing prompts in production. The eval harness must be fast enough to run per pull request, which caps golden-set size and argues for a tiered set: a fast tier in CI, a full tier nightly. --- ## ADR-012 — Rebuild indexes on failover rather than replicate them **Status** Accepted **Context** RTO is 4 hours. Cross-region replication of vector and lexical indexes roughly doubles their cost. **Decision** Replicate the object store and the metadata database. Rebuild vector and lexical indexes in the DR region from the derived zone on failover. **Consequences** Failover is slower but within RTO. The rebuild path must be exercised, not assumed — measured quarterly, and the decision reverts to replication if rebuild time approaches the RTO.