# Enterprise Generative Search — Azure and Open Source

**Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 41 views · 22 architecture decisions · Microsoft Azure and open source**

A generative search platform is not an LLM in front of a search box. It is a retrieval system with a language model attached to the end of it, and almost every failure it has is a retrieval, authorisation or evidence failure wearing a model's clothes. This package architects one for 45,000 employees over 40 million documents drawn from Microsoft 365, Confluence, ServiceNow, a 12 TB scanned archive, governed Databricks tables and 40 external domains — hybrid retrieval, a bounded agentic loop, claim-level grounding, and continuous evaluation as a release gate.

One decision carries the whole design: **the evidence contract.** Nothing enters the context window that the caller could not open themselves; every sentence carries an evidence id the retriever minted, or it is not shown; and retrieved content is data, never instruction. That contract is why the ACL fingerprint is part of every cache key, why the model holds no index credential, why citations are bound rather than requested, why a poisoned document is a ranking problem instead of a compromise, and why the downstream CRM assistant needed no security design of its own.

The stack is Azure and open source. Azure AI Search carries the hybrid index; PostgreSQL carries the ledger, the Apache AGE graph, conversation state and Langfuse traces; LangGraph orchestrates; LiteLLM is the model gateway; BGE-M3, a bge cross-encoder reranker and an NLI verifier run self-hosted on AKS GPU; Azure OpenAI does the one job that needs a frontier model. Every one of those choices, and the alternatives it beat, is recorded in the Architecture Decision Record.

---

## What is here

| Path | Contents |
|---|---|
| `diagrams/index.html` | The landing page — 41 views grouped into seven acts, every format linked, **with the full Architecture Decision Record and technology selection rationale below the index** |
| `diagrams/*.html` | One self-contained page per view: the inlined diagram plus the reasoning it deliberately omits, with copy / PNG / PDF export |
| `diagrams/svg/*.svg` | The same 41 views as SVG with the diagram XML embedded — re-opens in diagrams.net fully editable |
| `diagrams/drawio/*.drawio` | draw.io native source |
| `docs/architecture-decision-record.md` | The same 22 decisions and 32 technology rows as markdown, for reading rather than browsing |
| `specs/views.json` | Diagram specifications — the source of truth for every view (assembled from `part-a..g.json`) |
| `specs/manifest.json` | Acts, page titles, subtitles and the decision / assumption / risk cards (assembled from `manifest-a..b.json`) |
| `specs/adr.json` | The decision register and the technology selection table — the source of both ADR renderings |
| `scripts/build.sh` | Rebuilds every deliverable from the specs |
| `scripts/adr.mjs` | Renders the ADR onto the landing page and into `docs/` |
| `scripts/pin-icons.mjs` | Pins one mark per component so the same box carries the same icon on every page |
| `ask.md` | The original requirement |

895 icons are embedded in the files, so the deliverable renders identically anywhere with no network request. Each view page links to its own SVG and draw.io source, back to the index, and to the previous and next view.

To rebuild after editing a spec:

```bash
node scripts/pin-icons.mjs     # only after adding new component labels
bash scripts/build.sh          # Node 20+ and nothing else
```

Edit `specs/part-a.json` … `part-g.json` and `specs/manifest-a.json` / `manifest-b.json`; the build script assembles `views.json` and `manifest.json` from them and fails if the two lists disagree, if a view names a section that does not exist, or if the two orders diverge.

---

## The seven acts

| Act | What it lands | Views |
|---|---|---|
| 1 · Context and scope | The boundary, the evidence contract, and how a question is routed before a model is called | 01–04 |
| 2 · People and journeys | Who the platform is for, and the three journeys whose worst moments the rest of the set answers | 05–08 |
| 3 · Structure | The layering rule, the deployable units, query understanding, the retrieval fabric, every interface, and the model portfolio | 09–15 |
| 4 · Data | Ingestion, storage zones, chunking, the provenance model, permission propagation and freshness | 16–21 |
| 5 · Runtime | One request end to end, the bounded loop, fusion, context assembly, verification, tools, multimodal and degradation | 22–29 |
| 6 · Operations | Where it runs, how a prompt ships, what is measured, what is traced, what it costs | 30–35 |
| 7 · Assurance | Trust zones, identity, six attacks on untrusted content, sensitive data, governance evidence, failure modes | 36–41 |

---

## The requirement's demands, answered

The requirement sets out ten capability areas, twelve architectural principles and five categories of non-functional requirement. These are the places where the answer is a design decision rather than a product fact.

| Requirement | The answer here | View |
|---|---|---|
| Query planning that decides which sources to use | Six query classes, each with a retrieval plan, a generation mode and a budget, decided before any model is called. The agentic path is a routing outcome, not the default — and it costs 45 times a lookup. | 04 |
| Hybrid retrieval rather than vector-only | Four candidate generators fused with reciprocal rank fusion, then a cross-encoder. Dense-only measured Recall@50 of 0.71 against 0.92 fused, failing almost entirely on identifiers — so the lexical arm is load-bearing, not optional. | 12, 24 |
| Permissions must travel with the data | ACLs are extracted with the content, normalised to Entra object ids, and enforced as an index-side filter evaluated before scoring. Post-filtering is banned; deny beats allow; the group set fingerprint is part of every cache key. | 20 |
| Citations and provenance native to the data model | A citation joins a claim to a retrieval result, never straight to a chunk — so an audit can ask not only what was cited but how it was found and at what rank. | 19 |
| Claim-level grounding and hallucination mitigation | The draft is segmented into claims, each aligned against the supplied evidence by an NLI model from a different family than the synthesiser, with numbers checked deterministically. Unsupported claims are re-retrieved once, then dropped. | 26 |
| Untrusted retrieved content treated as data | Structural fencing: evidence occupies a typed slot that is never read as instruction, only cited sources become links, and a tool request whose provenance is a document body is rejected before policy is consulted. | 25, 38 |
| Controlled tool and action invocation | The model names an intent; a guard compiles it against allowlisted semantic views with bound parameters and executes as the caller. No model-authored SQL reaches a database, and figures are rendered from rows rather than retyped. | 27 |
| Graceful degradation to conventional search | Five rungs, each a defined product behaviour with a user-visible statement. The classic Search API deploys with no model, no GPU and no vector index, and carries its own 99.95% target. | 29 |
| Continuous evaluation at several layers | 1,400 golden queries across six layers, run as release gates: Recall@50 ≥ 0.90, groundedness ≥ 0.95, citation correctness ≥ 0.97, task success ≥ 0.82, zero access leaks, cost within 10% of baseline. | 32 |
| A trace that says which stage was wrong | One trace id across seven stages, carrying evidence ids, rank positions, token counts and claim verdicts. The rank of the cited passage is the single most diagnostic number in the platform. | 34 |
| Latency, cost and freshness optimised explicitly | TTFT 1.2 s P50 / 2.5 s P95; 0.021 USD blended per answered query; freshness declared per source class from 5 minutes to 24 hours, with staleness published rather than hidden. | 21, 35 |
| Evaluation built in from day one | The golden set exists before launch and grows from real failures through the RAGOps loop, so a fix that is not testable tomorrow is treated as a patch. | 33 |

---

## Where this design departs from the obvious answer

- **Databricks tables are never indexed.** A figure that can be computed correctly on demand should not be embedded and left to go stale. Structured questions go to a governed semantic layer instead.
- **Six model tasks, six independent choices.** Treating the LLM as one decision is how a platform ends up paying frontier prices to classify a question. Synthesis is managed; the other five run on open weights on our own GPUs.
- **The secondary region is built, not replicated.** Azure AI Search does not replicate an index across regions, so the indexer writes both — which also means the failover path is exercised by normal ingestion rather than by an annual test.
- **The classic Search API is a product, not a fallback flag.** It is what makes the bottom of the degradation ladder a working search engine instead of an error page.
- **Chunking is architecture, not a parameter.** Seven strategies, one per content type, because a support article split in half retrieves the symptom without the fix.

---

## Assumptions this set makes

Stated on the cards of the views they affect, and repeated here because they are the first thing a reviewer should challenge:

- 62,000 queries a day, peak 34 per second, 9,000 daily active users, 40 M documents and about 320 M chunks.
- Microsoft Entra ID is the single identity provider, and every source system's permissions can be expressed as Entra object ids. A source that cannot is not onboarded until it can.
- SAP and other systems of record reach the platform through governed Databricks tables rather than by direct connection.
- Reserved PTU capacity for the synthesis model with pay-as-you-go spill; pure pay-as-you-go raises the blended cost per query from 0.021 to about 0.028 USD.
- A 60-second entitlement projection lag is acceptable for all but a small number of sources, which verify against the source before display.
- Retrieval and generation quality figures come from a golden set built on this corpus. They are the numbers to re-measure first, not to inherit.