Architecture Decision Record

Twenty-two decisions carry this architecture. Everything else in the set is a consequence of one of them. Each record below states the forces that made the decision necessary, what was chosen, what was rejected and why, and what the choice costs — because a decision recorded without its price is a preference, not an architecture. Read a record's Consequences before its Decision if you want to judge it honestly.

The decisions

ADR-001 — The evidence contract is the governing constraint

Status Accepted · Tags Principle, Security, Grounding

Context

A generative search system fails in three ways that a search engine cannot: it can show a passage the reader was never entitled to open, it can assert something no retrieved passage supports, and it can be instructed by the content it retrieves. All three are architectural, not behavioural, and none is fixed by prompt wording.

Decision

Three rules bind every component. One: nothing enters the context window that the caller could not open themselves, so authorisation is an input to retrieval rather than a filter on its output. Two: every sentence carries an evidence id minted by the retriever, or it is not shown. Three: retrieved content occupies a typed slot that is never read as instruction.

Considered and rejected

• Prompt-level guardrails — instruct the model to respect permissions, cite sources and ignore embedded instructions — Makes the security boundary a matter of wording, which cannot be tested, cannot be proved to an auditor, and degrades silently with every model change.
• Post-hoc filtering — generate first, then remove anything the caller should not have seen — The content has already entered the model's context; the leak has already happened, and the ranked set was computed over documents the caller cannot see.

Consequences

• Positive: the downstream CRM assistant needed no access model of its own, and any answer can be replayed and re-derived from its stored evidence ids months later.
• Positive: a poisoned document becomes a ranking problem rather than a compromise.
• Negative: claim-level binding costs about 180 ms and 0.0009 USD per answer, and is the largest single quality expense in the design.
• Negative: the contract erodes silently — a cache key missing its ACL fingerprint breaks it with no test failing, which is why three specific assertions exist to catch exactly that.

Drawn in views 03 The Evidence Contract · 20 Permission Propagation · 26 Grounding and Verification

ADR-002 — Separate planning, retrieval, generation and verification

Status Accepted · Tags Orchestration, Quality

Context

The dominant industry pattern is a single agent with tools that searches, reads and answers as it sees fit. It is faster to build and it makes latency unbounded, cost unpredictable, and the failure classification impossible — nobody can say whether a wrong answer came from retrieval, ranking, context or the model.

Decision

Four stages with typed contracts between them, executed by a LangGraph state machine. The model proposes retrieval; a facade executes it under policy; a budget governor decides when the loop stops; a verifier decides whether the answer ships.

Considered and rejected

• Single agentic loop with tool calling — Authorisation becomes prompt-shaped, cost becomes unbounded, and each stage cannot be measured or replaced independently.
• Classic linear RAG with no loop at all — Cannot answer multi-hop or comparative questions, which are 11% of traffic and the ones the analyst population judges the platform by.

Consequences

• Positive: each stage has its own evaluation, its own latency budget and its own failure alert — which is what makes view 34 diagnostic rather than descriptive.
• Positive: the model and the retrieval stack can be replaced independently.
• Negative: more services, more contracts, more deployment surface than a single agent would need.
• Negative: one backward edge — the verifier asking the planner to retrieve again — has to be bounded explicitly, or it becomes the unbounded loop this decision was meant to avoid.

Drawn in views 02 High-Level Architecture · 09 Layered Architecture · 23 The Bounded Agentic Loop

ADR-003 — Azure AI Search as the hybrid retrieval substrate

Status Accepted, review scheduled · Tags Retrieval, Azure, Buy

Context

The platform needs BM25 and dense vector search over 320 M chunks in one query, filterable on ACL arrays and classification, with faceting, synonym maps and analyzer control, at 34 queries per second with a 450 ms budget. It also needs security trimming to be an index-side filter rather than an application concern.

Decision

Azure AI Search at S3 with 12 partitions and 3 replicas. Hybrid retrieval with reciprocal rank fusion is native, ACL arrays are filterable fields evaluated before scoring, and the operational burden of a sharded search cluster stays with the platform provider.

Considered and rejected

• OpenSearch or Elasticsearch self-hosted on AKS — Fully open source and entirely capable, but a 320 M document cluster is a full-time operational commitment — shard rebalancing, JVM tuning, upgrade windows — for a six-engineer team whose scarce skill is retrieval quality, not cluster operations.
• Qdrant or Milvus for vectors plus OpenSearch for lexical — Two systems means two consistency stories, two ACL projections and a fusion layer that must reconcile two definitions of a document version. The correctness risk sits exactly where this design cannot afford it.
• pgvector on the existing PostgreSQL — Excellent to about 10 M vectors and outside its comfort zone at 320 M; it would also couple the query path to the ledger's availability.

Consequences

• Positive: hybrid search, security filters and index aliasing are one managed service, and the alias swap in view 31 is atomic.
• Negative: real vendor coupling. It is contained by the retriever facade in view 12, which every backend implements, so a migration is a new backend rather than a rewrite.
• Negative: cross-region index replication is not provided, which is why the indexer writes both regions in view 30.
• Review: revisit if corpus growth passes about 500 M chunks or if the index line exceeds 20% of run cost.

Drawn in views 12 Retrieval Fabric · 30 Deployment Architecture · 31 CI/CD and Index Migration

ADR-004 — Hybrid retrieval, never vector-only

Status Accepted · Tags Retrieval, Quality

Context

Vector search is the default assumption in most generative search designs. On this corpus it fails on precisely the queries that dominate traffic: part numbers, policy identifiers, product codes and exact names — the lookup class, 38% of all questions.

Decision

Four candidate generators — BM25, dense HNSW, learned sparse and graph expansion — fused with reciprocal rank fusion at k=60, then reranked by a cross-encoder. The lexical arm is load-bearing and is not a configuration option.

Considered and rejected

• Dense vector retrieval only — Measured Recall@50 of 0.71 against 0.92 for the fused set, with almost all the loss on identifier-bearing queries.
• Score normalisation instead of rank fusion — BM25 scores, cosine similarities and graph walk scores are not on a common scale, and normalising them fabricates a comparability that does not exist.

Consequences

• Positive: Recall@50 of 0.92 and nDCG@10 of 0.68 on the golden set.
• Negative: four backends mean four failure modes on the query path, which is why the degradation ladder in view 29 exists.
• Negative: fusion weights per source are a governed policy rather than a tuning knob, which adds process to a change an engineer could otherwise make in an afternoon.

Drawn in views 12 Retrieval Fabric · 24 Hybrid Retrieval and Fusion · 29 Degradation Ladder

ADR-005 — Self-hosted BGE-M3 for embeddings

Status Accepted · Tags Models, Open source, Cost

Context

Embeddings are computed for 320 M chunks once, for about 240,000 changed items a day, and for every query. The cost scales with corpus size and queries per second rather than with answer tokens, and the model must be pinned for the life of an index because changing it invalidates the whole index.

Decision

BGE-M3 served on Hugging Face Text Embeddings Inference on an AKS GPU pool, producing dense and learned sparse representations from one model. Batch embedding runs on preemptible spot nodes.

Considered and rejected

• Azure OpenAI text-embedding-3-large — Strong quality and about 3.4 times the cost at this volume, with a per-call dependency on the critical path and a version lifecycle we do not control for an artefact that must stay pinned for years.
• A smaller open model such as all-MiniLM — Cheaper and materially worse on multilingual content, which is 14 languages here.

Consequences

• Positive: one model produces both the dense and the sparse representation, so the lexical bridge needs no second model to run or evaluate.
• Positive: the embedding version is ours to pin, and a provider deprecation cannot force an unplanned reindex.
• Negative: we now operate GPU nodes, with capacity planning, driver upgrades and spot interruption handling.
• Negative: about 4,100 USD a month of standing GPU cost across embedding, reranking and verification.

Drawn in views 15 Model Portfolio · 18 Chunking and Representation · 30 Deployment Architecture

ADR-006 — Cross-encoder reranking at depth 50

Status Accepted · Tags Retrieval, Quality, Latency

Context

Fusion produces a good candidate ordering and a poor top-10. The passages that end up in the context window are chosen from the top of that list, so ranking quality translates almost directly into groundedness.

Decision

bge-reranker-v2-m3 as a cross-encoder over the top 50 fused candidates, self-hosted on the same GPU pool as embeddings. Depth 50, not 200.

Considered and rejected

• Azure AI Search semantic ranker alone — Useful and cheap, and it cannot be evaluated or tuned against our golden set the way a model we host can. It remains the fallback.
• Reranking depth of 200 — The nDCG gain was inside the noise band on the golden set; the added latency was not.
• No reranking, fusion order only — nDCG@10 falls from 0.68 to 0.61, and the passages the model is handed get measurably worse.

Consequences

• Positive: 0.07 nDCG@10 for 110 ms and 0.0006 USD — the best quality-per-millisecond trade in the whole pipeline.
• Negative: a GPU dependency on the query path, mitigated by rung 2 of the degradation ladder, which serves fusion order and records a quality flag.

Drawn in views 15 Model Portfolio · 24 Hybrid Retrieval and Fusion · 29 Degradation Ladder

ADR-007 — LangGraph for orchestration

Status Accepted · Tags Orchestration, Open source

Context

The answer path is a bounded state machine with conditional edges, a hard budget, checkpointing and a requirement that every transition appears in a trace. It is not a linear chain and it is not an open-ended agent.

Decision

LangGraph in Python, with checkpoints in PostgreSQL, running as a service on AKS. Graph nodes map one-to-one onto the stages in view 09 and onto spans in the trace.

Considered and rejected

• Semantic Kernel — A capable orchestrator, and its planner abstractions pull towards the agent-decides-everything shape that ADR-002 rejects. It would be the stronger choice for a .NET estate.
• Azure AI Agent Service — Fastest to a working demo and it puts retrieval, tools and the loop inside a managed boundary, which is exactly where this design needs to enforce its own policy.
• Bespoke orchestration code — Perfectly viable, and it re-implements checkpointing, retries and streaming for no gain over a library the team already knows.

Consequences

• Positive: state, retries and streaming come for free, and the graph is the same artefact the trace shows.
• Negative: a fast-moving open-source dependency on the critical path; the version is pinned and upgrades run through the same evaluation gates as a prompt change.

Drawn in views 09 Layered Architecture · 10 Container Architecture · 23 The Bounded Agentic Loop

ADR-008 — LiteLLM as the model gateway

Status Accepted · Tags Models, Open source, Cost

Context

Six model tasks call four different providers — Azure OpenAI, self-hosted vLLM, self-hosted TEI and Azure AI Vision. Each needs routing, fallback, per-tenant budgets, token accounting and a way to move a task between providers without a code change.

Decision

LiteLLM proxy on AKS as the single egress point for every model call. No service holds a model name in its code; the gateway maps task to model, enforces budgets and emits token and cost telemetry per request.

Considered and rejected

• Direct provider SDKs in each service — Model names spread through the codebase, and cost accounting becomes a reconciliation exercise against a bill rather than a per-request fact.
• Azure API Management as the model gateway — Strong on quota and keys, weak on model-aware concerns — token accounting, semantic caching, per-task fallback. APIM stays at the edge, where those are not needed.

Consequences

• Positive: a provider deprecation becomes a configuration change plus an evaluation run — the whole migration plan in ADR-009.
• Positive: per-query cost is a measured fact rather than a monthly estimate, which is what makes the cost gate in view 32 possible.
• Negative: one more hop on the critical path, about 6 ms, and a component that must be as available as the models behind it.

Drawn in views 10 Container Architecture · 15 Model Portfolio · 35 Cost Model

ADR-009 — Managed frontier models for synthesis, open weights for everything else

Status Accepted · Tags Models, Cost, Azure

Context

Treating the LLM as one decision is how a platform ends up paying frontier prices to classify a question. The six model tasks have different quality requirements, different cost curves and different rates of change.

Decision

Azure OpenAI for answer synthesis, where quality requirements move faster than we can chase them. Open-weight models self-hosted for intent, rewriting, embedding, reranking and verification, where the workload is high-volume, well-defined and stable.

Considered and rejected

• One frontier model for every task — Would raise per-query model cost from 0.0161 to about 0.049 USD for no measured quality gain on the small tasks.
• Open weights for synthesis as well — Evaluated and close on the explanatory class, still behind on the multi-hop and comparative questions that carry the analyst population. It is the fallback and is re-evaluated each quarter.

Consequences

• Positive: 0.014 USD of the 0.0161 blended model cost is synthesis; everything else is rounding.
• Negative: a dependency on a provider's roadmap for the most visible component, mitigated only by the gateway abstraction and by keeping a tested open-weight fallback warm.
• Negative: PTU capacity has to be reserved before go-live, which is a commercial commitment made on a traffic forecast.

Drawn in views 15 Model Portfolio · 29 Degradation Ladder · 35 Cost Model

ADR-010 — Apache AGE on PostgreSQL for the knowledge graph

Status Provisional · Tags Data, Open source

Context

Multi-hop questions need entity and relationship expansion, and version supersession needs to be a first-class edge so a retired document can be retrieved and labelled rather than quietly outranked. The graph is a few hundred million edges, read-mostly, and reached only after a first retrieval pass.

Decision

Apache AGE, the open-source graph extension, on the PostgreSQL Flexible Server that already holds the ledger. openCypher queries, two-hop cap, bounded fan-out.

Considered and rejected

• Neo4j — The better graph database by some distance, and it adds a licensed product, a cluster and a second operational model for a component whose contribution to answer quality is not yet proven.
• Azure Cosmos DB with Gremlin — Managed and scalable, with a query model the team does not use elsewhere and a cost profile that does not suit a read-mostly workload of this size.
• No graph at all — Supersession and entity expansion would have to be simulated with metadata filters, which handles version chains poorly and multi-hop expansion not at all.

Consequences

• Positive: no new operational surface — it is an extension on a server that already exists.
• Negative: AGE is materially less mature than Neo4j, and its query planner is weaker on deep traversals. The two-hop cap is as much a containment as a design choice.
• Provisional: the graph arm's contribution is measured separately in view 32. If it does not pay for itself within two quarters it is removed, and if it proves central it is promoted to a purpose-built store.

Drawn in views 12 Retrieval Fabric · 19 Evidence Data Model · 21 Freshness Lifecycle

ADR-011 — Structured retrieval through an allowlisted semantic layer

Status Accepted · Tags Data, Security, Tools

Context

Nine per cent of questions are numeric or aggregate, and answering them from indexed text would mean embedding figures that go stale. Natural language to SQL is also the least reliable component available and the most dangerous one to get wrong.

Decision

The model names an intent and parameters. A SQL guard compiles that intent against views a data owner published, with bound parameters and a row cap, and executes as the caller so Unity Catalog grants apply. No model-authored SQL ever reaches a database, and figures are rendered from returned rows rather than retyped by the model.

Considered and rejected

• Model-generated SQL with a read-only account — Read-only prevents damage, not disclosure or a wrong answer; and a hallucinated join produces a number nobody can trace.
• Indexing the tables as text — Freezes figures at index time and reintroduces staleness into exactly the answers where staleness is least acceptable.

Consequences

• Positive: the executed statement is stored with the answer, so a disputed number is re-run rather than argued about.
• Negative: the semantic layer becomes a dependency of answer quality — a renamed view breaks a class of question, so the allowlist is version-pinned and its owners are on the change list.
• Negative: questions outside the allowlist cannot be answered numerically at all, which is a deliberate and visible limit.

Drawn in views 14 Source Connector Matrix · 27 Structured Retrieval and Tools · 38 Untrusted Content and Prompt Injection

ADR-012 — Permissions projected at index time, enforced as an index-side filter

Status Accepted · Tags Security, Retrieval

Context

Permissions must travel with the data through the entire retrieval pipeline. There are three ways to do it, and two of them are wrong in ways that are hard to see from the outside.

Decision

Source ACLs are extracted with the content, normalised to Entra object ids, and written onto every chunk as acl_allow and acl_deny arrays. At query time the resolved group set becomes a filter the search service evaluates before scoring. Deny is evaluated before allow.

Considered and rejected

• Post-filtering — retrieve, then drop what the caller cannot see — The ranked set was computed over documents the caller cannot see, so the visible top-k is quietly wrong even when nothing leaks.
• Checking entitlement against the source system per result — Correct and far too slow at retrieval depth 80 across four backends; it is retained only for the few sources where a 60-second projection lag is unacceptable.

Consequences

• Positive: authorisation is a filter, which means it is testable, fast, and applied before ranking.
• Negative: an entitlement change takes up to 60 seconds to become true in the index — a real window, stated rather than hidden.
• Negative: the platform now holds a projection of every source's permission model, which must be kept correct and is itself sensitive.

Drawn in views 14 Source Connector Matrix · 20 Permission Propagation · 37 Identity and Authorisation

ADR-013 — The ACL fingerprint is part of every cache key

Status Accepted · Tags Security, Cost, Caching

Context

Caching is the single largest cost lever available — 35% of queries repeat closely enough to serve from cache, and a cache hit costs no tokens at all. It is also the fastest way to turn an authorisation system into a leak with a hit rate.

Decision

Every cache key includes a SHA-256 fingerprint of the caller's resolved group set alongside the normalised query. Two people with different entitlements never share a cached answer, whatever they asked.

Considered and rejected

• Cache by query text alone — The highest-severity defect this platform can have: an answer built from documents the second caller cannot open.
• Cache only public-classified content — Safe and it discards most of the benefit, because the repeated questions are internal ones.

Consequences

• Positive: a 35% hit rate that is safe to enable, worth roughly 40% of the token line.
• Negative: hit rate falls as entitlement sets fragment — a person in an unusual set of groups effectively has a private cache.
• Negative: an entitlement change must invalidate by principal as well as by document, which is why the ACL delta job in view 20 does both.

Drawn in views 11 Query Understanding · 20 Permission Propagation · 35 Cost Model

ADR-014 — Claim-level binding verified by a separate NLI model

Status Accepted · Tags Grounding, Quality

Context

Asking a model to cite its sources produces citation-shaped strings whose accuracy is unmeasured. Citation correctness is a stated requirement at 0.97, and a requirement that cannot be measured cannot be met.

Decision

The draft is segmented into claims; each claim is aligned against the supplied evidence by a cross-encoder NLI model from a different family than the synthesiser; numbers and dates are checked deterministically. A claim with no supporting evidence is re-retrieved once, then dropped or abstained on.

Considered and rejected

• Trusting model-written citations — A citation the model composes is a string it can invent, and measurement shows it does.
• Azure AI Content Safety groundedness detection alone — A good managed signal at answer level, and it does not bind a specific sentence to a specific passage, which is what an auditor and a careful reader both need. It is retained as the fallback.

Consequences

• Positive: measured hallucination rate of 1.1% against a 1.5% ceiling, and citations that anchor to a passage rather than a document.
• Negative: 180 ms and 0.0009 USD per answer, plus a second model to operate and evaluate.
• Negative: over-refusal becomes a real risk, which is why false refusal rate is itself a release gate.

Drawn in views 03 The Evidence Contract · 26 Grounding and Verification · 32 Evaluation Harness

ADR-015 — Structure-aware chunking, one strategy per content type

Status Accepted · Tags Data, Quality

Context

Chunking decides what can be retrieved at all, and no single setting fits a policy document, a support article, a spreadsheet and a video transcript. A support article split in half retrieves the symptom without the fix.

Decision

Seven strategies, one per content type, each declaring its unit of meaning, its segmentation rule, the context header that travels with it, and its retrieval representation. Overlap applies to prose only.

Considered and rejected

• Fixed 512-token splitting with overlap everywhere — Measured 0.06 lower Recall@50, and it duplicates evidence in tables and articles, inflating apparent corroboration.
• Model-driven semantic chunking of the whole corpus — Attractive on quality and it costs a model call for every one of 40 M documents, and it re-runs on every reprocessing.

Consequences

• Positive: retrievable units correspond to units of meaning, and every chunk carries the version and heading path that make it interpretable alone.
• Negative: seven strategies to maintain and evaluate, and a new content type is a design task rather than a configuration change.
• Negative: a chunking change invalidates the whole index, so it is always a shadow build with an alias swap.

Drawn in views 16 Knowledge Ingestion Pipeline · 18 Chunking and Representation · 31 CI/CD and Index Migration

ADR-016 — One PostgreSQL for the ledger, the graph, conversation state and traces

Status Accepted, review scheduled · Tags Data, Operations

Context

Four workloads need a transactional store: the document and provenance ledger, the AGE graph, conversation state, and Langfuse traces. Each could justify its own service, and each would bring its own backup, patching, network and on-call story.

Decision

One PostgreSQL Flexible Server with zone-redundant high availability, four schemas, one operational model. The ledger is the only workload with an RPO.

Considered and rejected

• A managed service per workload — Four services, four failure modes and four bills for a platform whose transactional volume is modest — this is not a scale problem yet.
• Putting traces in Azure Monitor only — Excellent for infrastructure signals and poor at the LLM-specific structure — prompts, evidence sets, claim verdicts — that the trace in view 34 depends on.

Consequences

• Positive: one backup, one patching window, one network path, one set of credentials.
• Negative: a noisy-neighbour risk between trace writes and ledger reads, contained by separate connection pools and monitored per schema.
• Review: split traces out first if write volume or retention pressure grows; the ledger is the workload that must never be moved for convenience.

Drawn in views 10 Container Architecture · 17 Storage Zones · 30 Deployment Architecture

ADR-017 — Langfuse self-hosted for LLM tracing, Azure Monitor for everything else

Status Accepted · Tags Observability, Open source

Context

The question the platform must answer about any wrong answer is which stage was at fault. That needs a trace whose spans carry evidence ids, rank positions, token counts, claim verdicts and prompt versions — structure that generic APM tooling does not model.

Decision

OpenTelemetry instrumentation throughout, with LLM spans to a self-hosted Langfuse on the existing PostgreSQL and infrastructure signals to Azure Monitor, Prometheus and Grafana. Traces store identifiers, never passage text.

Considered and rejected

• Azure Monitor and Application Insights alone — Would force LLM structure into custom dimensions and lose the evidence-level detail that makes failure classification possible.
• A commercial LLM observability SaaS — Would send prompts and evidence outside the tenant, which conflicts directly with the sensitive-data controls in view 39.

Consequences

• Positive: one trace id spans all seven stages in view 34, and 90 days of traces is about 60 GB.
• Negative: another self-hosted component to run and upgrade.
• Negative: storing identifiers rather than text makes some debugging slower; prompt capture exists but is opt-in and time-boxed, deliberately.

Drawn in views 10 Container Architecture · 34 Observability and Tracing · 39 Sensitive Data and Tenant Isolation

ADR-018 — Evaluation is a release gate, built before launch

Status Accepted · Tags Quality, Process

Context

A generative search system with no golden set cannot distinguish an improvement from a regression, which makes every change after launch a guess. Building evaluation afterwards means the first six months of changes are unmeasured.

Decision

A 1,400-query golden set across six layers, run in the pipeline with hard thresholds that block a release. Retrieval and generation are scored separately. Access safety has a zero-tolerance gate with no waiver path. Cost is gated alongside quality.

Considered and rejected

• Post-hoc dashboards and manual spot checks — Detects regressions after users do, and provides no mechanism to stop one shipping.
• End-to-end scoring only — Cannot say whether the answer was wrong because the document was never found or because the model ignored it — the distinction the whole set is organised around.

Consequences

• Positive: a prompt change and a code change take the same path, and a quality regression cannot reach production unnoticed.
• Negative: about two annotator-days a month to maintain reference answers, plus the compute for a nightly full run.
• Negative: gates can be gamed by a golden set that drifts towards what the system is already good at, which is why it grows from real failures and is reviewed quarterly.

Drawn in views 31 CI/CD and Index Migration · 32 Evaluation Harness · 33 RAGOps Loop

ADR-019 — The classic search API is a separately deployable product

Status Accepted · Tags Availability, Product

Context

Generative answers depend on a model, a GPU pool, a vector index and a verifier. Every one of those is a dependency the search results themselves do not have, and an availability target that includes all of them is lower than one that does not.

Decision

A ranked-results Search API that deploys with no model, no GPU and no vector dependency, carrying its own 99.95% target against 99.9% for generative answers. It is the floor of the degradation ladder and a product in its own right.

Considered and rejected

• One endpoint with a generative flag — Couples the availability of the simple path to the dependencies of the complex one, which is the opposite of what a degradation ladder needs.
• Returning an error when generation is unavailable — Discards a working product to protect a feature.

Consequences

• Positive: the bottom rung of the ladder is a useful product rather than an error page, and the availability difference is priced deliberately.
• Negative: two surfaces to document, version and keep behaviourally consistent.

Drawn in views 13 Integration Surface · 29 Degradation Ladder · 36 Security Zones

ADR-020 — Airflow on AKS for ingestion orchestration

Status Accepted · Tags Data, Open source, Operations

Context

Ingestion is six connectors, four parse paths, backfills, per-source retries, priority by freshness class and a 14-hour full rebuild. It needs dependency-aware scheduling, backfill semantics and per-task observability, and it runs on the same cluster as its workers.

Decision

Apache Airflow with the Kubernetes executor on the existing AKS cluster, workers autoscaling from 4 to 60 pods on queue depth.

Considered and rejected

• Azure Data Factory — Strong for managed data movement and awkward for custom Python parse and embed steps, which is what most of this pipeline is.
• Azure Logic Apps or Durable Functions — Suited to event-driven glue rather than to a dependency graph with backfills and per-source priority.

Consequences

• Positive: backfill and per-task retry come as standard, and the workers run beside the models they call.
• Negative: Airflow is an operational commitment of its own — an upgrade path, a metadata database and a scheduler to keep healthy.

Drawn in views 10 Container Architecture · 16 Knowledge Ingestion Pipeline · 21 Freshness Lifecycle

ADR-021 — A warm secondary region with a dual index build

Status Accepted · Tags Availability, Azure, Cost

Context

The recovery target is 30 minutes, and a full index rebuild takes 14 hours. Azure AI Search does not replicate an index across regions, so a secondary must be built rather than copied.

Decision

The indexer writes both regions on every ingestion run. PostgreSQL replicates asynchronously to a promotable replica, and AKS in the secondary runs minimal and scales on failover. Search returns first, generative answers second.

Considered and rejected

• Rebuild on demand in the secondary — 14 hours against a 30-minute RTO.
• Active-active across both regions — Roughly doubles run cost for a workload whose availability requirement is 99.9%, and adds a cross-region consistency problem the design does not otherwise have.

Consequences

• Positive: the failover path is exercised continuously by normal ingestion rather than annually by a test.
• Negative: a standing cost for an event that may never happen, accepted explicitly against the RTO.
• Negative: it does not protect against a regional Azure OpenAI capacity event; the gateway holds deployments in two regions, which is a mitigation, not a guarantee.

Drawn in views 17 Storage Zones · 29 Degradation Ladder · 30 Deployment Architecture

ADR-022 — Self-hosted GPU for embedding, reranking and verification

Status Accepted · Tags Models, Cost, Operations

Context

Three model tasks run on every query or every chunk. Their cost scales with queries per second and corpus size, not with answer tokens, and their quality requirement is stable — which is the opposite profile to synthesis.

Decision

One AKS GPU pool, always on, for online embedding, reranking and verification; a second preemptible spot pool for batch embedding, so a reindex can never take capacity from the query path.

Considered and rejected

• Managed per-call endpoints for all three — About 3.4 times the cost at this volume, and a per-call dependency on the critical path for tasks that never change.
• One shared GPU pool for online and batch — A reindex would compete with live queries at exactly the moment both matter.

Consequences

• Positive: about 4,100 USD a month for three always-on model services, and 62% saved on the batch embedding path by using spot.
• Negative: GPU capacity planning, driver upgrades and spot interruption handling become the team's problem.
• Negative: a GPU pool outage degrades two rungs of the ladder at once, which is why rungs 2 and 3 are exercised together in game days.

Drawn in views 15 Model Portfolio · 29 Degradation Ladder · 30 Deployment Architecture

Technology selection, component by component

Every capability in the platform, the technology chosen for it, whether that technology is an Azure service or open source, why it won, and what was considered against it. Where a row names an ADR, the reasoning is above in full.

Experience and access

Orchestration and models

Retrieval and storage

Knowledge pipeline

Assurance and operations