Architecture Decision Record
Solution Architecture v1.0 · Azure-native, multi-tenant, sovereign-capable · Data & AI Global Practice · 2026-09
AI Executive Office — CXO Assistant Platform · Solution Architecture v1.0 · Azure-native, multi-tenant, sovereign-capable · Data & AI Global Practice · 2026-09
The argument these decisions serve is summarised in the Architecture One-Pager.
Thirty-two decisions that make up this architecture. Everything else on these thirty views is convention, and convention needs no defending. Each record is written to teach as well as to record: it opens with the forcing question and the context that makes it hard, states the decision so it can be checked, then shows the concrete Azure mechanism that realises it. It weighs the credible alternatives — some rejected, some genuinely right for an organisation with different constraints — and states what the choice buys, what it costs, the conditions that would flip it, and the principle that transfers to systems that are not this one. Decisions are grouped into nine areas; use the filter to read one area at a time.
Status of this document. This is a design, not a post-mortem of a running system. No figure in it is a measured production number: targets are engineering commitments to be tested, and volumes are stated assumptions drawn from the requirement. Azure service capabilities, SKU availability and regional service lists change, and several decisions here — most importantly the sovereign-region model availability in ADR-29 — must be re-verified against current Microsoft documentation for the target region before they are relied upon commercially. Where a decision rests on a customer fact not yet confirmed, the record says so rather than assuming in the platform's favour. The sovereign jurisdiction is anonymised throughout: “Meridia” and the region “Meridia Central” are placeholders for the customer’s actual country and Azure region, and “NISA” stands in for its national information-security authority. Substitute the real names, and re-check the real region’s service list, before this document is used commercially.
How to read a record
- Question: The forcing question: why a decision was needed at all.
- Context: The requirement, the scale and the constraint that make it hard.
- Decision: What this architecture does, stated so it can be checked.
- How it works on Azure: The concrete mechanism: which service, configured how.
- Options weighed: Chosen, rejected, deferred, or right elsewhere, with the reason for each.
- Consequences: What the choice buys and what it costs, both kept visible.
- Choose differently when: The conditions that would flip the decision for your system.
- Lesson: The principle that transfers beyond this platform.
Decision map
Foundations: The four commitments everything else is built on, and the ones a reviewer should attack first.
- ADR-01 · The decision record is the system of record, not the conversation
- ADR-02 · The model never touches data; a typed tool plane does
- ADR-03 · Deterministic before generative: the model never produces a number
- ADR-04 · Every claim is bound to evidence, and abstention is a first-class outcome
Experience and access: Where an executive meets the platform, and how a request gets in.
- ADR-05 · Teams is the primary surface; the web app is the deep surface
- ADR-06 · One front door: Front Door for the edge, API Management for the contract
- ADR-07 · Simple questions are synchronous; investigations are durable orchestrations
Orchestration and agents: What plans a turn, what runs it, and where it is hosted.
- ADR-08 · Azure AI Foundry Agent Service for the runtime, with the plan owned in our code
- ADR-09 · Specialist agents are capability packs in one runtime, not separate applications
- ADR-10 · Two model classes, routed — and a budget on every turn
- ADR-11 · Azure Container Apps for hosting, with AKS as a documented exit
Grounding and knowledge: How documents become citable evidence without leaking.
- ADR-12 · Hybrid retrieval with the security filter applied before scoring
- ADR-13 · Permissions and sensitivity are captured at ingestion and carried on the chunk
- ADR-14 · The semantic cache key includes the caller's permission fingerprint
Data and semantics: The analytical foundation, the shared model, and who is allowed to compute a number.
- ADR-15 · Microsoft Fabric as the analytical foundation, with shortcuts instead of copies
- ADR-16 · A canonical model with resolved entities, not query federation
- ADR-17 · One measure definition, owned by finance, shared with Power BI
- ADR-18 · A property graph for cross-domain relationships
- ADR-19 · Forecasts and anomaly scores come from registered models, not prompts
- ADR-20 · Freshness is declared per source and published with every answer
Decision and execution: Turning a recommendation into a governed action, and knowing whether it worked.
- ADR-21 · One execution plane holds every write credential in the estate
- ADR-22 · Options with cost, risk and delay — not a single recommendation
- ADR-23 · The loop closes on the record it opened
- ADR-24 · Proactive detection is deterministic and scheduled
Multi-tenancy: How one product serves many organisations without becoming many products.
- ADR-25 · Isolation is a purchased tier, not an engineering compromise
- ADR-26 · Configuration, not custom code — but the configuration is engineered
Sovereignty and security: Residency, key custody, and the authority chain from person to row.
- ADR-27 · Residency enforced by policy, network and key custody
- ADR-28 · The caller's authority reaches the row; no read-everything identity exists
- ADR-29 · Regional model availability is carried as a contracted risk, not an assumption
Operations and assurance: Releasing safely, degrading honestly, and proving it afterwards.
- ADR-30 · Evaluation is a release gate, and a prompt is a release artefact
- ADR-31 · Evidence is snapshotted, and the writer cannot delete
- ADR-32 · A written degradation contract, and one place the platform fails closed
Technology by capability
What each capability is built from, the credible alternative, and why this one. Every row links to the record that argues it. Origin labels: Azure · platform is a general Azure service; Azure · AI is part of the AI platform; This design means the requirement is met by a pattern the team builds rather than a product it buys.
| Capability | Choice | Origin | Credible alternative | Why this one | Record |
|---|---|---|---|---|---|
| Executive surface | Teams app (Azure Bot Service) + React web on Container Apps | Azure · platform | Copilot Studio, standalone portal only | Executives already live in Teams for approvals; the web app carries the depth Teams cards cannot. | ADR-05 |
| Global edge and WAF | Azure Front Door Premium | Azure · platform | Application Gateway + Cloudflare | Anycast edge, WAF and Private Link to an internal-mode gateway in one managed service. | ADR-06 |
| API gateway | API Management Premium, internal VNet mode | Azure · platform | Container Apps ingress + custom middleware | Per-tenant quota, token budgets and one enforced contract per surface, without writing a gateway. | ADR-06 |
| Application hosting | Azure Container Apps | Azure · platform | AKS, App Service, Functions only | A handful of stateless services with spiky, briefing-shaped load. AKS is a documented exit, not a rewrite. | ADR-11 |
| Long-running investigations | Durable Functions | Azure · platform | Service Bus + custom state machine | An investigation is a resumable orchestration with checkpoints, not a long HTTP request. | ADR-07 |
| Agent runtime | Azure AI Foundry Agent Service with Semantic Kernel in the host | Azure · AI | LangGraph, AutoGen, a hand-rolled loop | Managed threads, tool calling, tracing and evaluation in the same governance and network boundary as the models. | ADR-08 |
| Language models | Azure OpenAI — a reasoning model for planning and composition, a small model for routing and classification | Azure · AI | One large model for everything; open-weight models on managed compute | Most turns need routing, not reasoning. Two model classes is the single biggest lever on both latency and cost. | ADR-10 |
| Enterprise retrieval | Azure AI Search — hybrid vector + BM25, semantic ranker, index-side security filters | Azure · AI | pgvector on PostgreSQL, Elasticsearch, a vector-only store | Executive questions mix concepts with exact identifiers, and the security filter must run before scoring. | ADR-12 |
| Embeddings | Azure OpenAI text-embedding-3-large | Azure · AI | A smaller embedding model, or a domain-tuned one | In-region deployment alongside the other models, and no separate residency argument to make. | ADR-12 |
| Content safety and grounding | Azure AI Content Safety — prompt shields, groundedness detection, protected material | Azure · AI | A second LLM as judge, or open-source guardrail libraries | Managed, in-boundary, and versioned separately from the generating model, which is what makes the check independent. | ADR-04 |
| Analytical foundation | Microsoft Fabric — OneLake, Lakehouse, Warehouse, Data Factory, Real-Time Intelligence | Azure · platform | Synapse + ADLS, Azure Databricks | One capacity, one storage layer, one security model, and shortcuts to the customer's existing lake instead of copying it. | ADR-15 |
| KPI definitions | Power BI semantic model with DAX measures and row-level security | Azure · platform | dbt metrics, a hand-built metric service over SQL | The finance team already owns it, and the platform and existing BI must not disagree about a number. | ADR-17 |
| Cross-domain relationships | Azure Cosmos DB for Apache Gremlin | Azure · platform | Recursive SQL, a triple store, no graph at all | The flagship question is a multi-hop traversal over supplier, contract, purchase order and project. | ADR-18 |
| Forecasting and anomaly detection | Azure Machine Learning managed online endpoints | Azure · platform | Asking the language model; Fabric Data Science notebooks in-line | A forecast needs a registered, versioned, testable artefact with an interval — three things a prompt cannot provide. | ADR-19 |
| Decision and audit store | Azure SQL Database — elastic pool when pooled, single database when siloed | Azure · platform | Cosmos DB NoSQL, PostgreSQL Flexible Server | The decision record is relational, transactional and needs row-level security, point-in-time restore and a strong backup story. | ADR-01 |
| Evidence custody | Azure Blob Storage with an immutability policy and legal hold | Azure · platform | Keeping evidence in the database; a third-party WORM archive | Immutability enforced by the storage service, not by application logic, so an application compromise cannot rewrite history. | ADR-31 |
| Conversation state and semantic cache | Azure Cache for Redis Enterprise | Azure · platform | State in the decision store; no cache | Turn state is ephemeral and hot; the cache key includes the caller's permission fingerprint, which is what makes it safe. | ADR-14 |
| Batch and change-data ingestion | Fabric Data Factory pipelines with a self-hosted integration runtime | Azure · platform | Azure Data Factory standalone, Logic Apps for ingestion, third-party ELT | Same capacity, same lineage, and the self-hosted runtime is what reaches a ministry data centre. | ADR-20 |
| Event ingestion | Fabric Eventstream over Event Hubs | Azure · platform | Event Grid, Service Bus topics | High-throughput, replayable, and lands in the same lake as everything else. | ADR-20 |
| Action queue | Azure Service Bus, peek-lock with dead-lettering | Azure · platform | Storage queues, a database-backed outbox | An approved action must survive a crash exactly once. Peek-lock plus an idempotency key is what makes that true. | ADR-21 |
| Execution plane | Logic Apps Standard, one connector identity per target system | Azure · platform | Custom connector services, direct calls from the agent | Hundreds of enterprise connectors, and a blast radius bounded to one system per identity. | ADR-21 |
| Identity | Microsoft Entra ID — OBO flow, Conditional Access, PIM, managed identities | Azure · platform | A privileged service principal with application-side filtering | The AI must inherit the caller's authority. A read-everything identity makes every application bug a data breach. | ADR-28 |
| Keys and secrets | Azure Key Vault Managed HSM, customer-managed keys per tenant | Azure · platform | Standard Key Vault, platform-managed keys | The customer holding the key makes revocation a real control rather than a contractual promise. | ADR-27 |
| Catalogue, classification and lineage | Microsoft Purview | Azure · platform | A spreadsheet, or lineage inferred from pipeline metadata | Sensitivity labels captured at ingestion are what the retrieval filter and the egress DLP both act on. | ADR-13 |
| Networking | Hub-and-spoke VNet, private endpoints on every PaaS service, Azure Firewall Premium, ExpressRoute | Azure · platform | Service endpoints, public endpoints with IP restrictions | One controlled egress path is what makes an exfiltration claim defensible. | ADR-27 |
| Observability | Azure Monitor, Application Insights, OpenTelemetry GenAI traces, Foundry tracing | Azure · platform | A third-party APM, or logs only | A turn must be readable end to end — plan, tool calls, model version, grounding verdict — without leaving the boundary. | ADR-32 |
| Security posture and SIEM | Microsoft Defender for Cloud and Microsoft Sentinel | Azure · platform | The customer's existing SIEM with log forwarding | Native signal from the AI and data services, and reading a decision record is itself an audited event. | ADR-31 |
| Landing zone and governance | Management groups, Azure Policy, Bicep, Azure Deployment Environments | Azure · platform | Terraform, manual subscription setup | Residency and isolation are enforced by policy at the boundary, so a developer cannot deploy out of country. | ADR-27 |
| Tenant provisioning | Bicep-driven onboarding pipeline reading a tenant registry | This design | Manual provisioning, or a per-tenant repository fork | A tier is a deployment parameter. Onboarding tenant twelve must not cost what tenant two cost. | ADR-25 |
| Tool plane | Typed tool contracts behind API Management, resolved per role | This design | Direct database access from the agent; generated SQL | The one door between the model and the data, and therefore the one place authorisation can be audited. | ADR-02 |
| Decision engine | Situation → evidence → root cause → forecast → options → recommendation, as a service | This design | Prompting the model to produce a recommendation directly | Options with cost, risk and delay are what let an executive decide. A single recommendation asks them to trust instead. | ADR-22 |
| Proactive detection | Scheduled deterministic detectors over gold-zone measures and the graph | This design | An agent that continuously monitors the business | Repeatable, affordable and explainable. A model that watches is a model you cannot test. | ADR-24 |
| Tenant configuration | Versioned, schema-checked configuration artefacts in Git, promoted through rings | This design | Configuration rows in a database edited through an admin UI | "Configuration, not custom code" only holds if the configuration is engineered, reviewable and reversible. | ADR-26 |
| Evaluation | Golden-question suites per tenant, run as a release gate in the pipeline | This design | Manual spot checks, or vendor benchmark scores | Retrieval and generation are scored separately so a regression has a stage rather than a shrug. | ADR-30 |
The decisions, and the alternatives that lost
Foundations
The four commitments everything else is built on, and the ones a reviewer should attack first.
ADR-01 · The decision record is the system of record, not the conversation
Status: Accepted · Shown on views: 06, 13, 19, 30
When an executive approves an intervention through a conversational interface, what exactly has been created?
Context. The brief asks for auditability, closed-loop monitoring, and the ability to reconstruct why a decision was taken. A chat transcript satisfies none of these: it has no state machine, no outcome, no relationship to the KPI the intervention was meant to move, and no way to be reopened. The requirement also asks for a decision intelligence platform rather than a chatbot, and the difference has to be visible somewhere in the data model or it is only a slogan.
Decision. Situation, evidence, options, approval, action and outcome form one durable, append-only entity with an explicit state machine. The conversation is a projection of that entity, not the other way round. Every surface — Teams card, web thread, governance export — renders the same record.
How it works on Azure. Azure SQL Database holds the decision, evidence, action and outcome tables with row-level security by tenant and temporal history on state transitions. Evidence payloads are snapshotted to immutable Blob Storage and referenced by identifier. A decision is never updated in place: state changes are appended, which is what makes the timeline reconstructable four months later.
| Option | Verdict | Reasoning |
|---|---|---|
| Decision record as the durable entity | Chosen | Makes audit, reopening and outcome tracking properties of the model rather than features bolted on later. |
| Chat thread with structured annotations | Rejected | Cheaper to build and the default for conversational products, but there is no state machine, so "which decisions are awaiting me" becomes a search rather than a query. |
| Records only for actions above a value threshold | Rejected | The threshold is the argument. A decision not to act is exactly the one an auditor asks about. |
| Push the record into the customer's existing case system | Right elsewhere | Right where the customer already runs governance on a platform they trust. It costs the closed loop, because the outcome monitor then has no home. |
What it buys
- Audit, closed-loop monitoring and reopening all fall out of one model rather than three subsystems
- The Today view is a query, not an aggregation over chat history
- A decision survives the conversation, the session and the user's departure
What it costs
- A relational store with a real backup and retention obligation, which the analytical estate does not need
- Every surface must write through the decision service rather than talking to the model directly
- More upfront modelling before anything demonstrable exists
Choose differently when. Choose a lighter model when the assistant genuinely only answers questions and never proposes an action — an internal knowledge assistant with no execution path does not need this and should not pay for it.
Lesson. In any assistant that leads to action, the durable artefact is the decision, not the dialogue. Get that wrong and auditability becomes log archaeology.
ADR-02 · The model never touches data; a typed tool plane does
Status: Accepted · Shown on views: 02, 15, 16, 26
How does a language model get access to enterprise data without being given access to enterprise data?
Context. The requirement is explicit that AI must never have unrestricted access and must inherit the user's authorisation context. The tempting shortcut — give the model a database connection and let it write SQL — is fast to demonstrate and impossible to govern: the generated query is unbounded, the identity is the service's rather than the caller's, and nothing about the access is reviewable in advance.
Decision. All data access is through a catalogue of typed tools with declared inputs, outputs and required scopes. The model proposes which tool to call; the tool plane decides whether the caller may, executes the read, and mints an evidence identifier. The orchestrator holds no credential for any store.
How it works on Azure. Tools are published through API Management with a schema per operation and a policy that requires an on-behalf-of assertion. The tool implementation calls the semantic model, AI Search or the graph using the caller's exchanged token, so row-level security and index filters evaluate against the person rather than the service. Managed identity is used only for the platform's own stores, never for tenant business data.
| Option | Verdict | Reasoning |
|---|---|---|
| Typed tool contracts, caller identity carried through | Chosen | Authorisation becomes reviewable in advance, and every read is replayable for audit. |
| Text-to-SQL against a read replica | Rejected | Demonstrates in a day and fails review in an hour. Unbounded queries, no per-caller authority, and no way to know what a future prompt will produce. |
| Model Context Protocol servers per domain | Deferred | A good fit for the same shape and worth adopting as the ecosystem matures; the governance argument here is about typing and identity, which MCP does not by itself supply. |
| Pre-computed answer sets only | Right elsewhere | Right for a fixed executive dashboard with no drill-down, and considerably cheaper. It cannot answer a question nobody anticipated. |
What it buys
- Compromising the orchestrator yields the ability to ask questions as the caller, and nothing more
- Every read is typed, logged and replayable, which is what makes the audit trail meaningful
- Tool latency and failure rate become measurable per source
What it costs
- Every new question class may need a new tool, which is slower than a general query interface
- The tool catalogue becomes a governed artefact with its own change process
- Per-caller queries cache poorly, addressed in ADR-14
Choose differently when. Allow generated queries when the data is genuinely public within the organisation, the store is a disposable replica, and no answer leads to an action. Even then, budget for the query that scans everything.
Lesson. Give a model a capability, never a credential. The narrower the door, the easier it is to prove what came through it.
ADR-03 · Deterministic before generative: the model never produces a number
Status: Accepted · Shown on views: 02, 17, 19
When an executive asks for the cash position, what actually computes the figure?
Context. The requirement warns against asking the LLM to calculate financial KPIs when a deterministic engine can. The deeper problem is asymmetric consequence: a clumsy narrative is embarrassing and recoverable, whereas a wrong figure presented to a board is not. Executives also calibrate on the answers that turn out wrong, so a single fabricated number costs more trust than fifty good answers earn.
Decision. KPI values come from DAX measures in the governed semantic model. Forecasts, anomaly scores and risk scores come from registered ML endpoints with intervals. The language model classifies the question, plans the retrieval, frames the scenario and narrates the result. It never produces a figure, a forecast or a score.
How it works on Azure. The metric service is the only component permitted to query the Power BI semantic model, and it does so with the caller's identity so row-level security applies. Azure ML managed online endpoints serve forecasting and anomaly models as versioned artefacts. The routing table in view 17 is implemented as a classifier plus a dispatch map, and a test asserts that no other code path returns a numeric measure.
| Option | Verdict | Reasoning |
|---|---|---|
| Deterministic computation, generative narration | Chosen | Trust control and cost control in one decision: most questions route to a cheap path with a single small-model call. |
| Let the model compute from retrieved tables | Rejected | Works impressively in a demo with small tables and fails silently on aggregation, currency, period boundaries and null handling. |
| Model computes, then a checker verifies | Rejected | Two model calls to reach a number that a measure already defines, with a checker that shares the first model's blind spots. |
| Code interpreter in a sandbox | Deferred | Genuinely useful for ad-hoc what-if analysis on data already retrieved, and a candidate for a later scenario-modelling capability under the same evidence rules. |
What it buys
- Numbers are reproducible, testable against expected values and explainable without reference to a model version
- The dominant cost path is avoided for most questions
- A model upgrade cannot change a reported figure
What it costs
- A question with no corresponding measure cannot be answered until the measure is built
- Measure coverage becomes an onboarding workstream with its own backlog
- The platform will sometimes abstain where a competitor confidently guesses
Choose differently when. Relax this for exploratory analysis clearly labelled as such, on data the user already has, where no figure leaves the session. Never relax it for anything that reaches a decision record.
Lesson. Decide which of your outputs are opinions and which are facts, and route them to different machinery. A system that blurs the two will eventually be wrong in the expensive direction.
ADR-04 · Every claim is bound to evidence, and abstention is a first-class outcome
Status: Accepted · Shown on views: 04, 15, 29
What does the platform do when it cannot support part of an answer?
Context. The brief asks the system to say "I don't have sufficient evidence to answer this reliably" rather than fabricate. That is easy to write and hard to build, because it requires knowing which parts of a generated answer are supported — which means the binding between claim and evidence must be mechanical rather than requested politely in a prompt.
Decision. The composition step returns a claim-to-evidence map alongside the prose. Each claim is verified against the retrieved evidence by an independent check. Unsupported claims are removed and the gap is stated. Where the whole answer fails, the platform abstains in those words. Abstention rate is a monitored metric and is deliberately not driven to zero.
How it works on Azure. Azure AI Content Safety groundedness detection performs the check, versioned independently of the generating model so the checker is not marking its own work. Evidence identifiers are minted by the tool plane at read time and carried through composition. The verdict, the dropped claims and the abstention are all written to the decision record and surface in view 30's evidence pack.
| Option | Verdict | Reasoning |
|---|---|---|
| Mechanical claim binding with an independent groundedness check | Chosen | The only version of this that survives contact with a regulator, because the check does not depend on the model's own confidence. |
| Prompt the model to cite and to refuse when unsure | Rejected | Improves the average case and fails exactly when it matters, since a confidently wrong model is also confidently cited. |
| Show the sources and let the reader judge | Rejected | Transfers the verification burden to the person least able to carry it at 07:10. |
| Human review before every executive answer | Right elsewhere | Right in a small, very-high-stakes setting such as regulatory filings. It does not scale to a conversational front door and removes the reason to have one. |
What it buys
- Trust is earned by the answers that are declined as much as by the ones given
- The evidence pack in view 30 becomes producible, because binding already exists
- Grounding rate is a release-blocking signal rather than a sentiment
What it costs
- Added latency on every turn, which is the first thing challenged when the 5-second target is missed
- Answers are shorter and occasionally less satisfying than an ungrounded competitor's
- An additional service in the critical path and in the cost model
Choose differently when. A lower bar is reasonable for an internal brainstorming assistant where the reader is expected to verify. It is not reasonable anywhere an answer can become an approval.
Lesson. "Do not hallucinate" is not a control. A control is a component that can fail the output, run by something other than the thing that produced it.
Experience and access
Where an executive meets the platform, and how a request gets in.
ADR-05 · Teams is the primary surface; the web app is the deep surface
Status: Accepted · Shown on views: 01, 02, 04
Where does an executive actually meet this platform?
Context. Executive adoption is the largest product risk, and the real incumbent is a chief of staff rather than a competing tool. Anything requiring an executive to open a new application, remember a URL and learn a navigation model starts at a disadvantage that accuracy will not recover. Approvals in particular already happen in Teams for most Microsoft-estate customers.
Decision. The morning brief, notifications and approvals are delivered as Adaptive Cards in Teams. Investigation, option comparison, drill-down and the governance centre live in a web application that a card can deep-link into. Mobile shares the web contract rather than having its own.
How it works on Azure. Azure Bot Service fronts the Teams channel; Adaptive Cards carry the brief and the approval action, with the action posting to the same Experience API the web app uses. The web app runs on Container Apps behind Front Door. Single sign-on flows from the Teams identity, so an approval requires no second authentication.
| Option | Verdict | Reasoning |
|---|---|---|
| Teams for brief and approval, web for depth | Chosen | Meets the executive where approvals already happen without forcing a rich comparison view into a card. |
| Standalone portal only | Rejected | Cleanest to build and the surface most likely to go unopened after week three. |
| Everything in Teams | Rejected | An option set with cost, risk and delay, and a drill-down into evidence, do not fit an Adaptive Card without becoming unreadable. |
| Copilot Studio as the front end | Right elsewhere | A strong choice for a customer standardising on Microsoft 365 Copilot with lighter governance needs. It cedes control of the orchestration and evidence contract, which is the product here. |
What it buys
- Approvals happen where the executive already is, which is the difference between daily use and abandonment
- One identity, one session, no second sign-in for an approval
- Push and presence come free rather than being built
What it costs
- Two front-end surfaces to maintain and to keep consistent
- A dependency on the customer's Microsoft 365 estate and its policies
- Adaptive Card capability limits constrain what the brief can show
Choose differently when. Lead with the web app where the customer is not a Teams organisation, or where a regulator objects to decision content transiting a collaboration platform — the brief then becomes an email deep-link.
Lesson. For executive software, distribution beats features. The surface that is already open wins arguments that the better interface loses.
ADR-06 · One front door: Front Door for the edge, API Management for the contract
Status: Accepted · Shown on views: 02, 08, 21, 26
What enforces tenant quota, contract and public exposure, and where?
Context. A multi-tenant platform needs per-tenant rate limits and token budgets, one enforced API contract per surface, and no public endpoint on anything behind the perimeter. Building that into the application means every service re-implements it and the sovereign tenants get a different answer from the pooled ones.
Decision. Azure Front Door Premium terminates TLS, provides WAF and DDoS protection, and reaches the platform over Private Link. API Management in internal VNet mode is the only ingress to the application, and is where tenant quota, token budgets and contract validation are enforced.
How it works on Azure. Front Door Premium origins connect by Private Link to an internal-mode APIM. APIM policies validate the Entra token, extract the tenant claim, apply per-tenant rate and token quotas, and forward with claims to Container Apps. No data or AI service has a public endpoint; all are reached through private endpoints with private DNS.
| Option | Verdict | Reasoning |
|---|---|---|
| Front Door + internal APIM | Chosen | Edge protection and contract enforcement as managed services, with quota policy in one place for every tenant. |
| Application Gateway + custom middleware | Rejected | Cheaper at one region and re-implements quota, versioning and developer contract in application code. |
| Container Apps ingress directly | Rejected | Adequate for a prototype and leaves per-tenant throttling — the control that protects one tenant from another — in the application. |
| Third-party API gateway | Right elsewhere | Right where the customer already standardises on one and wants a single policy surface across their estate. |
What it buys
- A noisy tenant is throttled before it reaches shared compute or shared model quota
- One contract per surface, versioned, with the policy visible to security review
- No public endpoint anywhere behind the perimeter
What it costs
- APIM Premium is a material fixed cost, which weighs heavily on a small siloed tenant
- Another hop in the latency budget
- Two products to configure before the first request is served
Choose differently when. Drop APIM for a single-tenant deployment with no quota requirement and one client; the policy layer is then genuinely unearned cost.
Lesson. Put multi-tenant fairness controls in infrastructure, not in application code. The tenant you need to throttle is the one whose code path you are least keen to touch.
ADR-07 · Simple questions are synchronous; investigations are durable orchestrations
Status: Accepted · Shown on views: 02, 15, 16
What happens when a question genuinely takes ninety seconds to answer?
Context. The requirement distinguishes a simple KPI question, targeted under five seconds, from a complex investigation where an asynchronous experience is acceptable. Holding an HTTP request open for ninety seconds fails at every proxy, loses all work on a restart, and gives the user a spinner instead of progress.
Decision. Turns needing at most four tool calls run synchronously. Anything larger is promoted to a durable orchestration with checkpoints, progressive disclosure of intermediate findings, and a resumable identifier the executive can leave and return to.
How it works on Azure. Durable Functions runs the investigation as an orchestration with activity checkpoints in its own state store, so a host restart resumes rather than restarts. Findings stream to the client over Server-Sent Events as each stage completes, and the same identifier is addressable later from the Teams card or the web thread.
| Option | Verdict | Reasoning |
|---|---|---|
| Threshold-based promotion to a durable orchestration | Chosen | The user waits only when waiting is short, and long work is crash-safe and resumable. |
| Everything synchronous | Rejected | Simple and correct until the first cross-system question, which is the question the product exists for. |
| Everything asynchronous | Rejected | Uniform and honest, but a five-second question behind a job-status poll feels broken. |
| Service Bus plus a hand-written state machine | Rejected | The same design, built rather than bought, with retries and compensation to write and test. |
What it buys
- Long investigations survive a deployment or a host failure
- Progressive disclosure makes ninety seconds feel like work rather than a hang
- The promotion threshold is a tuning knob rather than a rewrite
What it costs
- Two execution paths to build, test and observe
- The client must handle streaming and resumption
- Orchestration state is another store to secure and to reason about for residency
Choose differently when. Stay synchronous throughout if every question in your domain is bounded by a single system call. Go fully asynchronous if none of them are.
Lesson. Choose the interaction model from the work's distribution, not its average. The tail is what the user remembers.
Orchestration and agents
What plans a turn, what runs it, and where it is hosted.
ADR-08 · Azure AI Foundry Agent Service for the runtime, with the plan owned in our code
Status: Accepted · Shown on views: 02, 07, 16
Build the agent runtime, or adopt one?
Context. Agent frameworks are abundant and change quickly. The parts that matter here — tool calling with the caller's identity, threads, tracing, evaluation, and staying inside a private network boundary in a sovereign region — are infrastructure. The part that is genuinely ours is the planning policy: which capability answers which question, under what budget, with what fallback.
Decision. Adopt Azure AI Foundry Agent Service for thread management, tool invocation, tracing and evaluation. Keep the planning policy, the budget enforcement and the tool authorisation in our own orchestrator host, so the platform's distinctive behaviour is not a framework's default.
How it works on Azure. The orchestrator runs on Container Apps using Semantic Kernel for plan composition and calls Foundry Agent Service for agent execution. Tracing follows OpenTelemetry GenAI conventions into Application Insights, and Foundry's evaluation harness is reused for the release gate in ADR-30. All of it sits behind private endpoints in the tenant's network.
| Option | Verdict | Reasoning |
|---|---|---|
| Foundry Agent Service + our planner | Chosen | Buys the plumbing and the governance surface; keeps the routing policy, which is where the product's judgement lives. |
| Fully hand-rolled orchestration loop | Rejected | Maximum control, and a year of building threads, tracing and evaluation that a managed service already provides in-boundary. |
| LangGraph or AutoGen on Container Apps | Right elsewhere | Excellent frameworks and the right answer for a team already fluent in them or targeting more than one cloud. Here they add a residency and support argument for capability the platform already has. |
| Copilot Studio orchestration | Rejected | Fast to a demonstration; cedes the tool authorisation and evidence contract that this platform is fundamentally about. |
What it buys
- Tracing and evaluation exist from day one rather than being retrofitted
- Everything stays inside one network and residency boundary
- The routing policy is testable code we own
What it costs
- A dependency on a fast-moving managed service, including its regional availability
- Framework upgrades become a release concern
- Some capability duplication between the framework and our planner
Choose differently when. Build your own when you need portability across clouds, or when your orchestration is genuinely unusual. Adopt more of the platform when your team is small and your governance needs are ordinary.
Lesson. Buy the plumbing, build the judgement. The framework is not your product; the routing policy might be.
ADR-09 · Specialist agents are capability packs in one runtime, not separate applications
Status: Accepted · Shown on views: 02, 07, 16
Is a finance agent a deployment, or a configuration?
Context. The requirement names ten specialist domains and describes them as modular capabilities rather than independent LLM applications. It also asks for an agent marketplace as a product module. Those two only reconcile if adding a capability is cheap, and it is only cheap if it is not a deployment.
Decision. A specialist agent is a prompt pack plus a tool catalogue plus an evaluation set, versioned with the tenant configuration and loaded into one shared runtime. Agents return structured results to the orchestrator and never call each other.
How it works on Azure. Packs are configuration artefacts in the tenant configuration repository, schema-checked in the pipeline and loaded by the orchestrator host at start-up and on change. The tool catalogue per pack resolves to API Management operations the caller's role permits. Foundry threads keep per-agent context without a per-agent deployment.
| Option | Verdict | Reasoning |
|---|---|---|
| Capability packs in a shared runtime | Chosen | Adding a domain is configuration, which is what makes a marketplace a feature rather than a programme. |
| One microservice per agent | Rejected | Attractive for team autonomy at large scale; here it multiplies deployments, network policy and cost by ten for no isolation benefit. |
| One monolithic prompt covering every domain | Rejected | Cheapest, and it degrades as domains are added because every question pays for every domain's instructions. |
| Free agent-to-agent negotiation | Rejected | Where latency and cost hide. Structured returns to a single orchestrator keep a turn bounded and traceable. |
What it buys
- A new domain is a configuration change with an evaluation set, not a release
- A tenant can be behind on a pack without being behind on the platform
- One runtime to secure, observe and scale
What it costs
- Packs share a blast radius: a bad shared runtime release affects every domain
- Per-domain resource limits must be enforced in the budget rather than by isolation
- Pack versioning becomes its own governance surface
Choose differently when. Split into services when a domain needs genuinely different scaling, a different compliance boundary, or an independent team release cadence.
Lesson. Modularity is a property of the configuration model, not of the deployment topology. Ten deployments are not ten modules; they are ten things to patch.
ADR-10 · Two model classes, routed — and a budget on every turn
Status: Accepted · Shown on views: 16, 17, 23
Which model answers, and what stops a single question costing a fortune?
Context. Most executive turns are routing and narration over a computed figure. A minority are genuine multi-step investigations. Sending everything to a reasoning model makes the common case slow and expensive, and an agentic system with unbounded fan-out has unpredictable unit economics — which breaks per-tenant pricing before it breaks the latency target.
Decision. A small, fast model performs intent classification, entity resolution and routing. A reasoning model is used only for planning an investigation and for final composition. Every turn carries an explicit budget for tokens, tool calls and wall-clock, set by the orchestrator before any agent runs.
How it works on Azure. Both deployments are Azure OpenAI in the tenant's region, with provisioned throughput sized for the briefing spike and pay-as-you-go overflow. API Management enforces the per-tenant token quota; the orchestrator enforces the per-turn budget and degrades to a narrower answer rather than exceeding it. Cost per interaction is exported to Fabric and alerted on per tenant.
| Option | Verdict | Reasoning |
|---|---|---|
| Routed two-class models with per-turn budgets | Chosen | The largest single lever on both latency and cost, and it makes unit economics predictable enough to price. |
| One capable model for everything | Rejected | Simplest to operate and reason about, and it pays reasoning-model prices for "what is our cash position?". |
| Open-weight models on managed compute | Deferred | Worth revisiting for the classification path where volume is high and the task is narrow; it adds an operational burden that is hard to justify at prototype scale. |
| No budget, monitor and react | Rejected | The first runaway investigation is discovered on the invoice, and by then a tenant has been affected. |
What it buys
- Predictable cost per interaction, which a commercial model can be built on
- The common path is fast because it never touches the expensive model
- A runaway turn degrades gracefully instead of consuming the tenant's quota
What it costs
- Two deployments to manage, evaluate and keep pinned
- The router itself can be wrong, sending a hard question down a cheap path
- Budget exhaustion is a user-visible state that must be explained well
Choose differently when. Use a single model when volume is low enough that operational simplicity beats unit cost, or when routing errors are more damaging than latency.
Lesson. In an agentic system, cost is an architectural property, not an operational surprise. Decide the budget before the first token, or discover it on the invoice.
ADR-11 · Azure Container Apps for hosting, with AKS as a documented exit
Status: Accepted · Shown on views: 08, 21
What runs the application, given a mid-size tenant and a small platform team?
Context. The workload is a handful of stateless HTTP services with a pronounced daily spike around the morning brief and long quiet periods. The platform team runs many tenants and cannot afford per-tenant cluster operations. The requirement lists Container Apps, AKS and App Service as candidates without choosing.
Decision. Azure Container Apps for all application services, with scale rules on concurrent requests. AKS is recorded as the migration target if per-tenant scale, custom networking or a service mesh becomes necessary, and services are kept container-native so that migration is a hosting change rather than a rewrite.
How it works on Azure. One Container Apps environment per tenant runtime, VNet-integrated, with managed identity per app and Dapr left off until something needs it. Scale from three replicas to thirty on concurrency. Durable Functions runs on an Elastic Premium plan for the orchestration workload, which Container Apps does not serve as well.
| Option | Verdict | Reasoning |
|---|---|---|
| Container Apps | Chosen | Right-sized for the workload and the team; no cluster to operate per tenant. |
| AKS | Right elsewhere | The correct answer at large scale, with mesh requirements or a platform team that already runs Kubernetes well. Here it is an operational burden per tenant with no matching benefit. |
| App Service | Rejected | Mature and familiar, with weaker container and scale-to-zero characteristics for a spiky, multi-service workload. |
| Functions for everything | Rejected | Excellent for the orchestration and event paths, which is where it is used. A poor fit for long-lived streaming connections to the client. |
What it buys
- No Kubernetes operations per tenant, which is what makes twenty tenants affordable
- Scale-to-low between briefings materially reduces the pooled-tier bill
- Deployment is an image reference, which keeps the release pipeline simple
What it costs
- Less control over networking and scheduling than AKS
- Platform limits may eventually bind, hence the recorded exit
- Two hosting models — Container Apps and Functions — to operate
Choose differently when. Start on AKS when you already run it well, when you need a service mesh or custom CNI, or when a single tenant's scale justifies a dedicated cluster.
Lesson. Choose the hosting platform for the team that will operate it at 3 a.m., not for the architecture diagram. Record the exit so the choice stays reversible.
Grounding and knowledge
How documents become citable evidence without leaking.
ADR-12 · Hybrid retrieval with the security filter applied before scoring
Status: Accepted · Shown on views: 14, 26, 27
How is the document estate searched without returning something the caller may not read?
Context. Executive questions mix concepts with exact identifiers — a contract number, a project code, a supplier name. Vector search alone handles identifiers poorly. Separately, the obvious implementation of permissions — retrieve, then filter the results — leaks nothing but silently returns a worse top-k, because the documents the caller may read were crowded out before filtering and never scored.
Decision. Retrieval is hybrid: vector and keyword, fused, then reranked. The caller's group membership is applied as a filter inside the search query, evaluated before scoring, so the top-k is drawn only from documents the caller may read.
How it works on Azure. Azure AI Search with a vector field from text-embedding-3-large alongside a searchable text field, combined by reciprocal rank fusion and reranked by the semantic ranker. Security identifiers captured at ingestion are stored as a filterable field, and every query carries an OData filter built from the caller's Entra group claims. Retrieve fifty, rerank to eight.
| Option | Verdict | Reasoning |
|---|---|---|
| Hybrid retrieval, filter inside the query | Chosen | Correct top-k and correct permissions, which are two problems that look like one. |
| Vector-only retrieval | Rejected | Fine for conceptual questions and weak exactly where executives are precise — contract numbers and project codes. |
| Retrieve then filter in the application | Rejected | The failure nobody notices: no leak, and a quietly worse answer that looks fine. |
| An index per security group | Rejected | Correct and unmanageable; group membership changes daily and index count explodes. |
What it buys
- Identifier-shaped queries work, which is most of what an executive types
- Permissions cannot be bypassed by an application bug in the result-handling code
- Top-k is honest, so citation quality is measurable
What it costs
- Filter cardinality affects query performance on large indexes
- The security field must be maintained accurately at ingestion, which ADR-13 addresses
- Two retrieval modes and a reranker to tune and evaluate
Choose differently when. Vector-only is fine for a corpus with no access differentiation and no identifiers. Post-filtering is acceptable only when every user can read everything, which is rarely true and never true in government.
Lesson. Authorisation belongs inside the query, not after it. A filter applied to results protects the data and corrupts the ranking.
ADR-13 · Permissions and sensitivity are captured at ingestion and carried on the chunk
Status: Accepted · Shown on views: 14, 26, 30
Where does the index learn who may read a document?
Context. Re-deriving permissions at query time means calling the source system for every candidate document, which is slow and creates a dependency on a system that may be down. Deriving them once and forgetting them means the index becomes wrong the moment a permission changes. Sensitivity labels have the same problem and additionally drive the egress controls.
Decision. Source access-control identifiers and Purview sensitivity labels are captured during ingestion and written as fields on every chunk. They are never re-derived at query time. Documents are superseded rather than deleted, so a citation made four months ago still resolves to the version that was cited.
How it works on Azure. The crawl reads permissions through Microsoft Graph for SharePoint content and the source API elsewhere, writing group identifiers into a filterable collection field on each chunk. Purview classification runs at landing and its label is stored alongside. Re-crawl every four hours; high-sensitivity libraries are crawled more often, and content labelled restricted is re-checked against the source at query time, at a stated latency cost.
| Option | Verdict | Reasoning |
|---|---|---|
| Capture at ingestion, carry on the chunk | Chosen | Fast queries and a bounded, stated exposure window rather than an unbounded unstated one. |
| Re-derive at query time for everything | Rejected | Always correct, always slow, and it makes retrieval unavailable whenever the source is. |
| Trust a single-level document classification | Rejected | Coarse enough to be either over-restrictive or wrong; executive material is precisely where per-document permissions matter. |
| Push-based permission change notifications | Deferred | The right long-term answer where the source supports change subscriptions, and worth adopting per source as capability allows. |
What it buys
- Query latency stays flat regardless of corpus size
- Retrieval keeps working when the source system does not
- Sensitivity is available to both the retrieval filter and the egress DLP
What it costs
- A permission drift window of up to one crawl interval, which must be stated to the tenant rather than glossed
- Ingestion becomes more complex and more failure-prone
- Superseding rather than deleting increases storage
Choose differently when. Re-derive at query time when the corpus is small, permissions change constantly, or the exposure window is legally unacceptable — and accept the latency openly.
Lesson. Copying an authorisation decision creates a staleness window. You cannot remove it; you can only measure it, shorten it for the material that matters, and tell people it exists.
ADR-14 · The semantic cache key includes the caller's permission fingerprint
Status: Accepted · Shown on views: 15, 27
How does a per-caller authorisation model coexist with a cache?
Context. ADR-27's rule — every read carries the caller's identity — makes results caller-specific, and caller-specific results cache badly. Caching on the question text alone is the obvious optimisation and is a data breach with a hit rate: two executives asking the same question are entitled to different answers.
Decision. Cached entries are keyed on the normalised question, the tenant, the model and prompt version, the data-freshness stamp, and a hash of the caller's effective permission set. Two callers share a cache entry only when they would have been entitled to identical results.
How it works on Azure. Azure Cache for Redis Enterprise holds the entries. The permission fingerprint is a stable hash of the caller's Entra group claims relevant to the sources involved, recomputed per session. Entries are invalidated on model or prompt version change and on the freshness stamp advancing, so a cached answer can never be stale in a way the answer does not admit.
| Option | Verdict | Reasoning |
|---|---|---|
| Permission-fingerprint cache key | Chosen | Keeps most of the benefit — executives in the same role do share entries — with no possibility of a cross-authority hit. |
| Cache on question text only | Rejected | The highest hit rate and an authorisation bypass, which is not a trade-off worth having. |
| No cache at all | Rejected | Safe and needlessly expensive during the morning briefing spike, when many executives ask overlapping questions. |
| Cache only the retrieval, not the answer | Deferred | A useful middle ground worth measuring once real query distributions exist. |
What it buys
- Real hit rates during the briefing spike, where identical questions cluster
- A permission change naturally invalidates by changing the fingerprint
- Model and prompt version in the key means an upgrade cannot serve stale reasoning
What it costs
- Lower hit rate than a naive cache, and highly variable by tenant
- The fingerprint must be computed correctly or the whole guarantee is void
- Another store inside the residency boundary
Choose differently when. Cache on content alone only where every user has identical entitlements — a public corpus, or a single-role deployment.
Lesson. A cache key is an authorisation statement. Anything the answer depended on belongs in the key, and permissions are the thing most often left out.
Data and semantics
The analytical foundation, the shared model, and who is allowed to compute a number.
ADR-15 · Microsoft Fabric as the analytical foundation, with shortcuts instead of copies
Status: Accepted · Shown on views: 08, 11, 12
What holds the analytical data, given that most customers already have a lake?
Context. Every target customer has an existing warehouse, lake or BI estate. Copying it into the platform doubles the storage bill, creates a second version of the truth, and makes the platform responsible for data it did not produce. The requirement asks for integration with existing lakes and BI, not their replacement.
Decision. Microsoft Fabric provides OneLake, the lakehouse, the warehouse, pipelines and real-time ingestion on one capacity with one security model. The customer's existing lake is mounted as a OneLake shortcut wherever the storage supports it, rather than copied.
How it works on Azure. A Fabric capacity per tenant — dedicated F-SKU for siloed, a workspace on shared capacity for pooled. Medallion layout in OneLake, Data Factory pipelines for batch and change-data ingestion, Eventstream for events, and shortcuts to the customer's ADLS or supported external storage so their gold tables are readable without duplication.
| Option | Verdict | Reasoning |
|---|---|---|
| Fabric with shortcuts to existing storage | Chosen | One capacity to size and secure, and the customer's lake stays theirs instead of being copied. |
| Synapse + ADLS + separate Power BI | Rejected | More control over each component and three security models, three billing models and a lineage story to assemble by hand. |
| Azure Databricks | Right elsewhere | The stronger choice where the customer already runs Databricks or the workload is engineering-heavy. Here the semantic model and Power BI integration are the centre of gravity. |
| Read the source systems directly at query time | Rejected | No historical baseline, no cross-system joins, and every executive question becomes load on production ERP. |
What it buys
- One capacity, one storage layer, one security model, one lineage graph
- The customer's existing investment is read rather than duplicated
- The semantic model sits natively beside the warehouse that feeds it
What it costs
- Capacity sizing is a real per-tenant exercise and a real per-tenant cost floor
- A significant dependency on one product's roadmap
- Shortcut support varies by source storage, so some copies remain unavoidable
Choose differently when. Choose Databricks or a Synapse-style assembly where the customer's team already lives there, or where the engineering workload dominates the reporting workload.
Lesson. Prefer mounting to copying. A second copy of the truth is a second thing to reconcile, and reconciliation is where executive trust is lost.
ADR-16 · A canonical model with resolved entities, not query federation
Status: Accepted · Shown on views: 11, 12, 13
How does the platform answer a question that spans three systems that share no key?
Context. "Which suppliers are affecting projects that are already financially at risk?" requires supplier identity to be the same thing in the ERP, the procurement system and the portfolio tool. It is not: the same supplier is three records with three keys and often three spellings. Query federation joins tables; it does not resolve identity, and no amount of virtualisation supplies a key that does not exist.
Decision. A canonical model with explicit entity resolution for supplier, project, cost centre and contract. Conformed entities retain their source keys so a merge is reversible, and resolution runs as a pipeline stage with a human review queue for low-confidence matches.
How it works on Azure. Silver-layer notebooks in Fabric perform deterministic matching on identifiers and probabilistic matching on names and addresses, writing a crosswalk table with confidence scores. Matches below threshold enter a review queue surfaced in the tenant console. The crosswalk is the one table that is never cached downstream, because a merge is correct on approval and wrong for every consumer until they agree.
| Option | Verdict | Reasoning |
|---|---|---|
| Canonical model with entity resolution | Chosen | The only approach that makes the flagship cross-domain question answerable at all. |
| Query federation over source systems | Rejected | Avoids copying data and cannot join on a key that does not exist. It also puts executive queries onto production ERP. |
| Ask the language model to match entities | Rejected | Plausible-looking matches with no confidence score, no review path and no reversibility — the worst possible property for a supplier merge. |
| Buy a master data management platform | Right elsewhere | Right where the organisation needs MDM as an enterprise capability. It is a programme of its own, and this platform needs resolution only for its own model. |
What it buys
- Cross-domain questions become answerable, which is the product's central claim
- Reversible merges, because entity resolution is a judgement and judgements get revised
- Match confidence is visible, so an answer can state that identity is uncertain
What it costs
- The hardest and most underestimated workstream in the programme, and it is not an AI problem
- A human review queue means an operational process, not just a pipeline
- A bad merge propagates into every answer until it is corrected
Choose differently when. Federate when the systems already share a reliable enterprise key. Buy MDM when several programmes need resolution and this one would be building it for everyone.
Lesson. Cross-system intelligence is an identity problem wearing an integration costume. Solve identity first, or the joins will be confidently wrong.
ADR-17 · One measure definition, owned by finance, shared with Power BI
Status: Accepted · Shown on views: 11, 12, 17
Who owns the definition of a KPI the platform reports?
Context. The fastest way to destroy executive trust is a figure that differs from the board pack, because it is not obviously wrong — it just makes both numbers suspect. Building a metric layer inside the platform guarantees this happens, since two definitions of gross margin will diverge within a quarter no matter how carefully they start.
Decision. KPI definitions live once, as measures in the governed semantic model, owned by the customer's finance function. The platform reads them; it does not restate them. Where the customer already owns a definition in their BI estate, the platform adopts it rather than writing its own.
How it works on Azure. A Power BI semantic model over the Fabric warehouse holds DAX measures with row-level security. The platform's metric service is the only component that queries it, using the caller's identity. Power BI reports read the same model, so a report and an answer cannot disagree. Measure changes go through the same versioned promotion as any other artefact.
| Option | Verdict | Reasoning |
|---|---|---|
| One semantic model, shared with the customer's BI | Chosen | Makes disagreement structurally impossible rather than a thing to police. |
| A metric service over SQL, owned by the platform | Rejected | More portable and it recreates the exact divergence the platform exists to eliminate. |
| dbt metrics or a similar semantic layer | Right elsewhere | A good choice where the customer's stack is already dbt-centred and Power BI is not the reporting front end. |
| Let each agent compute its domain's measures | Rejected | Ten definitions of headcount within a year, and no way to say which is right. |
What it buys
- A platform answer and a board pack cannot disagree
- Finance owns the definition, which is both correct and politically necessary
- Row-level security is defined once and inherited by every consumer
What it costs
- A hard dependency on the semantic model's availability and refresh schedule
- Measure coverage becomes an onboarding gate, and gaps are visible as abstentions
- Change control on measures is slower than the platform team would like
Choose differently when. Build your own metric layer when there is no incumbent BI estate to align with, or when the incumbent's definitions are known to be wrong and replacing them is in scope.
Lesson. Never own a second definition of a number someone else already owns. The reconciliation meeting costs more than the integration.
ADR-18 · A property graph for cross-domain relationships
Status: Accepted · Shown on views: 11, 13, 17, 18
What answers a question that is three relationship hops deep?
Context. The flagship question traverses supplier to contract to purchase order to project to budget line, then asks which of those projects are already at financial risk. In a star schema that is a chain of joins that must be written in advance; in a graph it is a traversal that can be composed at query time. Detection also needs to propagate a supplier signal outward to the projects it affects, which is the same traversal in reverse.
Decision. A property graph holds the canonical entities and the relationships between them, rebuilt from the gold layer. It holds relationships and identifiers, never measures — figures always come from the semantic model.
How it works on Azure. Azure Cosmos DB for Apache Gremlin, rebuilt from the Fabric gold layer on each refresh, zone-redundant. Vertices carry canonical identifiers and a small set of attributes used for traversal; every numeric answer is fetched from the semantic model by identifier after the traversal returns. The graph is therefore fully rebuildable and carries no independent recovery objective.
| Option | Verdict | Reasoning |
|---|---|---|
| Property graph alongside the warehouse | Chosen | Multi-hop traversal and signal propagation become natural rather than a generated join chain. |
| Recursive SQL over the warehouse | Rejected | Avoids another store and becomes unreadable and slow past two or three hops, which is where the interesting questions start. |
| A dedicated graph database product | Right elsewhere | Stronger graph capability, and it adds a product outside the Azure managed estate with its own residency and support argument. |
| No graph; pre-compute the known paths | Rejected | Workable for a fixed question set and precisely wrong for a platform whose promise is questions nobody anticipated. |
What it buys
- Multi-hop questions are composable at query time rather than pre-written
- Detection can propagate a signal along real relationships
- Fully rebuildable, so it needs no backup story of its own
What it costs
- Another store to secure, size and keep in step with the warehouse
- Gremlin is a skill the team must actually have
- A rebuild lag means the graph can briefly disagree with the warehouse, which must be stated
Choose differently when. Skip the graph when relationships are shallow and stable — two hops over a well-modelled star schema does not need one.
Lesson. Add a graph when the questions are about relationships rather than aggregates. Keep the measures out of it, or you now have two places a number can come from.
ADR-19 · Forecasts and anomaly scores come from registered models, not prompts
Status: Accepted · Shown on views: 17, 18
What produces the forecast behind "what happens if we do nothing?"
Context. The requirement asks for forecasting, anomaly detection, risk scoring and scenario analysis, and explicitly separates LLM reasoning from statistical calculation. A forecast presented to an executive needs an interval, a method that can be described, and reproducibility — a language model asked to project a trend supplies a number with the appearance of all three and none of the substance.
Decision. Forecasting, anomaly detection and risk scoring are registered, versioned models served behind managed endpoints. Every prediction returns an interval and the model version. The language model frames the scenario and explains the result; it never generates the projection.
How it works on Azure. Azure Machine Learning holds the registry and serves managed online endpoints with at least two nodes. Features come from the gold layer's feature store. Model version accompanies every prediction into the decision record, so a forecast shown four months ago can be attributed to the model that made it.
| Option | Verdict | Reasoning |
|---|---|---|
| Registered models behind managed endpoints | Chosen | Versioned, testable, reproducible, and able to express uncertainty honestly. |
| Ask the language model to project the trend | Rejected | Fast and plausible, with no interval, no method and no reproducibility — three properties an executive forecast cannot do without. |
| Forecasting inside the semantic model | Rejected | Fine for simple time intelligence and unable to carry anomaly detection or multivariate risk scoring. |
| Fabric Data Science notebooks in-line | Right elsewhere | Right for exploration and for a prototype. It lacks the endpoint versioning and serving guarantees the record needs. |
What it buys
- Every projection carries an interval, which is what makes "confidence" a number rather than an adjective
- Model version is auditable alongside the decision that used it
- Statistical quality can be improved without touching the conversational layer
What it costs
- An ML lifecycle to run — training, registration, drift monitoring — alongside the AI lifecycle
- Endpoint cost even at low utilisation
- Cold-start models for a new tenant with no history, which must be declared rather than hidden
Choose differently when. Use simple statistical baselines in the semantic model when the forecast horizon is short and the series is well behaved. Do not use a language model either way.
Lesson. A forecast without an interval is an opinion with a decimal point. If the component cannot express uncertainty, it should not be producing the number.
ADR-20 · Freshness is declared per source and published with every answer
Status: Accepted · Shown on views: 09, 11, 12, 25
What does "current" mean when twelve source systems refresh at twelve different rates?
Context. The requirement notes that not every source needs real-time integration, which is true and incomplete: the consequence is that "what changed since yesterday?" means something different for a source on fifteen-minute change capture than for one on a nightly extract. Silence about that difference is the failure mode that destroys trust fastest, because the answer looks authoritative either way.
Decision. Every source declares a freshness contract at registration. The platform records the actual as-of timestamp per source, publishes it with every answer that used the source, and raises an alert when a source falls behind its contract. Zones are organised by rebuildability, so only the operational estate carries a real recovery objective.
How it works on Azure. Data Factory pipelines write an as-of watermark per source into a freshness register in the gold layer. The tool plane reads it and attaches it to every result, so the composition step can state it. Azure Monitor alerts when a watermark exceeds its contract, and view 25's degradation contract says what the answer does in that case: serve the last good value with the date shown, up to 24 hours, then withhold.
| Option | Verdict | Reasoning |
|---|---|---|
| Declared contracts, published as-of, alert on breach | Chosen | Makes staleness a visible property of an answer rather than an invisible property of the platform. |
| Present everything as current | Rejected | The default in most BI, and the reason executives quietly stop believing dashboards. |
| Real-time integration everywhere | Rejected | Unaffordable and unnecessary; several source systems cannot support it at all. |
| Refuse to answer whenever anything is stale | Rejected | Honest and useless — one late nightly feed would silence the whole morning brief. |
What it buys
- An executive can tell how much weight to put on a figure
- Stale-source incidents are detected by the platform rather than reported by the user
- Ingestion cost is spent where freshness is actually needed
What it costs
- Every answer carries more qualification, which takes design work to keep readable
- A freshness register to maintain per source and per tenant
- Some questions become unanswerable rather than answered approximately
Choose differently when. A single global freshness statement is enough when all sources genuinely refresh together. That is rare outside a single-system deployment.
Lesson. Publish the age of your data. An answer that hides how old it is has substituted confidence for accuracy.
Decision and execution
Turning a recommendation into a governed action, and knowing whether it worked.
ADR-21 · One execution plane holds every write credential in the estate
Status: Accepted · Shown on views: 02, 20, 26
What is actually permitted to change a record in an enterprise system?
Context. The requirement states that AI must never have unrestricted write access. The weaker reading — give the agent a write tool and guard it with a prompt — fails to the first prompt injection through a retrieved document. The strong reading is that the AI identity should hold no write scope at all, so that there is nothing to be talked into using.
Decision. Agents may only create action proposals. A separate execution plane, holding a distinct connector identity per target system, performs the write after a human approval, with an idempotency key and an authority re-check at execution time.
How it works on Azure. Approved proposals go to Azure Service Bus with peek-lock and dead-lettering. Logic Apps Standard workflows claim them and call the target system through its connector, each workflow running under its own user-assigned managed identity scoped to one system and one operation set. The external reference returned by the target system is written back to the decision record. The Azure OpenAI and orchestrator identities have no write role assignment anywhere.
| Option | Verdict | Reasoning |
|---|---|---|
| Separate execution plane, connector identity per system | Chosen | Prompt injection cannot reach a write path that the model has no credential for. |
| Write tools in the agent's catalogue, guarded by policy | Rejected | One identity away from a very bad afternoon, and the guard is code the model is trying to reason around. |
| One shared service principal for all writes | Rejected | Simpler to operate and makes the blast radius of one compromise the entire estate. |
| Emit a task for a human to perform manually | Right elsewhere | Right for a first release into a very conservative customer, and it forfeits the closed loop and most of the value. |
What it buys
- The AI's blast radius on write is exactly zero by construction, not by policy
- A compromise of one connector identity is bounded to one system
- Idempotency and retry are solved once, in one place
What it costs
- Another component in the path from approval to effect
- Connector identities and their scopes are a real ongoing governance burden
- Terminal-unknown outcomes must be reconciled against the target system rather than retried
Choose differently when. Collapse the plane into the application only where writes are to a system you fully own and a compromise is contained anyway. Never collapse it where the model can read untrusted content.
Lesson. The safest way to stop a model misusing a credential is not to give it one. Separate the thing that decides from the thing that acts.
ADR-22 · Options with cost, risk and delay — not a single recommendation
Status: Accepted · Shown on views: 04, 19, 20
What should the platform put in front of an executive at the moment of decision?
Context. The requirement is explicit: the executive should decide with evidence rather than simply receive an AI-generated recommendation. This is a product decision as much as an architectural one. A single recommendation asks for trust the platform has not earned and gives the executive nothing to reason with; three options with their trade-offs give them the thing they are actually paid to do.
Decision. The decision engine produces a situation summary, evidence, root cause, forecast, impact and an option set — each option carrying cost, risk and expected delay — plus a recommended option and its confidence. The executive may approve, reject, request more information, delegate or escalate, and a rejection is recorded as evidence too.
How it works on Azure. The decision service composes the option set from deterministic inputs: cost from the semantic model, risk from the ML risk endpoint, blast radius from the graph, precedent from retrieval. The language model orders and narrates them. Approval thresholds and dual-approval rules are tenant configuration evaluated by the policy engine, not code.
| Option | Verdict | Reasoning |
|---|---|---|
| Option set with quantified trade-offs | Chosen | Gives the executive something to decide with, and makes the platform's reasoning inspectable. |
| Single recommendation with a confidence score | Rejected | Simpler to present and asks for trust rather than earning it; a rejected recommendation also teaches the platform nothing. |
| Raw evidence, no recommendation | Rejected | Defensible and unhelpful — it recreates the problem of reconciling exports, which is what the platform is replacing. |
| Automatic execution below a threshold | Deferred | Explicitly anticipated in the requirement for low-risk actions, and correct once acceptance data exists to set the threshold honestly. |
What it buys
- The executive's judgement is supported rather than replaced, which is what gets the platform adopted
- Rejected options are retained, so an audit sees the alternatives too
- Confidence becomes a computed property of the inputs rather than a tone of voice
What it costs
- Generating credible options is much harder than generating a recommendation
- Cost, risk and delay must each come from somewhere defensible
- More screen space and more executive reading time per decision
Choose differently when. A single recommendation is right for high-volume, low-stakes decisions where the cost of reading three options exceeds the cost of being wrong.
Lesson. Decision support means supplying the trade-off, not the conclusion. The conclusion is the part the executive is accountable for.
ADR-23 · The loop closes on the record it opened
Status: Accepted · Shown on views: 19, 20, 23
How does anyone find out whether an intervention worked?
Context. The requirement asks for closed-loop monitoring — decision, action, outcome, KPI monitoring, escalate or recommend next. Most platforms stop at the action, which means the organisation learns nothing and the same intervention is proposed again next quarter with the same result.
Decision. Every executed action schedules an outcome watch on the KPI it was meant to move, over a window set per action class. The verdict — improved, unchanged, worsened or inconclusive — is written to the same decision record, and a failed intervention reopens that decision rather than starting a new one.
How it works on Azure. The decision service schedules the watch on execution. A Fabric job compares the KPI against the forecast that justified the action and writes the verdict. Reopening posts a new Teams card that links to the original record, so the executive sees what changed rather than a fresh, contextless alert.
| Option | Verdict | Reasoning |
|---|---|---|
| Outcome watch on the same record, with reopening | Chosen | The organisation accumulates evidence about which interventions actually work. |
| Stop at execution | Rejected | The common implementation, and it makes the platform an expensive way to send instructions. |
| A separate outcomes dashboard | Rejected | Data without a loop; nobody opens it, and it cannot reopen a decision. |
| Ask the executive afterwards whether it worked | Right elsewhere | Useful as a supplementary signal and worth capturing, but it is a memory, not a measurement. |
What it buys
- Recommendation quality becomes measurable rather than asserted
- A failed intervention escalates instead of being quietly forgotten
- Precedent for the option set in ADR-22 accumulates naturally
What it costs
- Attribution is genuinely hard — many things move a KPI in fourteen days
- Slow-moving public-sector outcomes will often return inconclusive
- A scheduling and state-management burden per executed action
Choose differently when. Skip the loop only where actions have no measurable effect on any tracked metric — in which case, ask why the action was recommended.
Lesson. "Inconclusive" must be a permitted verdict. A closed loop that can only report success is not measuring anything.
ADR-24 · Proactive detection is deterministic and scheduled
Status: Accepted · Shown on views: 17, 18, 23
What actually notices that something is going wrong?
Context. The requirement asks the platform to proactively surface exceptions, emerging risks, KPI deterioration and anomalies. The obvious implementation — an agent that continuously monitors the business — is unaffordable at any real data volume, non-deterministic, and impossible to test. The harder truth is that detection is not the difficult part: triage is. A brief with forty exceptions has surfaced none.
Decision. Detection is a scheduled deterministic pipeline: thresholds from tenant configuration, anomaly models, forecast deviation, and graph propagation from one domain's signal to another's entities. Language models narrate a signal a detector has already raised. Triage — materiality scoring, deduplication to one item per root cause, and expiring logged suppressions — decides what reaches the brief.
How it works on Azure. An hourly Fabric job scores gold-layer measures against baselines and calls the Azure ML anomaly endpoint; supply-chain and financial-anomaly classes are additionally event-triggered through Eventstream. Signals below the materiality bar are recorded rather than discarded, so a missed exception can be traced to the threshold that hid it. Suppressions are attributed, expiring records — never silent.
| Option | Verdict | Reasoning |
|---|---|---|
| Deterministic scheduled detection, generative narration | Chosen | Repeatable, testable, affordable, and explainable when an executive asks why they were not told. |
| An agent that continuously monitors | Rejected | Compelling in a demonstration; unbounded cost, no reproducibility, and no way to test that it would have caught something. |
| Alert on every threshold breach | Rejected | Detection without triage, which is how a brief becomes forty items and then zero readers. |
| Let executives configure their own alerts | Right elsewhere | A reasonable supplement for a power user, and it puts the burden of knowing what to watch on the person who hired the platform to know. |
What it buys
- Detection is testable against historical data — you can ask whether it would have caught last year's overrun
- Cost is a function of data volume rather than of model calls
- Suppression is visible and expiring, so it cannot quietly become a repealed control
What it costs
- Detectors must be authored and tuned per tenant, which is onboarding effort
- Novel failure modes nobody wrote a detector for are missed
- Threshold tuning is a recurring operational task, not a one-off
Choose differently when. Use a model-driven exploratory sweep where the failure modes are genuinely unknown and volumes are small enough to afford it — as a supplement to deterministic detection, never as a replacement.
Lesson. Detection is cheap and triage is the product. The measure of a proactive system is what it decided not to tell you.
Multi-tenancy
How one product serves many organisations without becoming many products.
ADR-25 · Isolation is a purchased tier, not an engineering compromise
Status: Accepted · Shown on views: 08, 10, 21
How isolated is one customer from another?
Context. The requirement asks for logical or physical isolation appropriate to each customer's security tier, and names a sovereign public-sector scenario. A single answer cannot serve both a commercial group that wants a low price and a ministry that will not share a database server with anyone. Choosing pooled-only loses the sovereign customers; choosing siloed-only makes the product unaffordable for everyone else.
Decision. Two tiers from one codebase. Pooled tenants share compute with a per-tenant search index, row-level security, per-tenant encryption keys and a per-tenant Fabric workspace. Siloed tenants get a dedicated subscription deployed from the same templates. Sovereign public-sector tenants are siloed by default. Changing tier is re-running the onboarding pipeline against a new subscription.
How it works on Azure. A shared control plane holds the tenant registry — tier, region, key references, configuration version — and never tenant business data. Bicep templates parameterised by tier deploy either a workspace in shared capacity or a full dedicated subscription. Pooled data separation is Azure SQL row-level security plus per-tenant customer-managed keys; siloed is a separate database, capacity and key vault.
| Option | Verdict | Reasoning |
|---|---|---|
| Two tiers, one codebase, tier as a deployment parameter | Chosen | Serves both commercial economics and sovereign requirements without maintaining two products. |
| Pooled only | Rejected | Best unit economics and it disqualifies the platform from the public-sector deals it was designed for. |
| Siloed only | Rejected | Simplest isolation story, and the fixed cost floor per tenant makes mid-market customers unsellable. |
| Separate Entra tenant per customer | Right elsewhere | Maximum separation, appropriate for the most sensitive deployments, at a significant operational cost the tier model already largely achieves. |
What it buys
- One product, one release, two shapes — so tenant twelve costs materially less than tenant two
- Sovereign requirements are met without special-casing the code
- The control plane can be operated by one team without access to any tenant's data
What it costs
- Every feature must be tested in both shapes, which roughly doubles the release matrix
- The pooled tier's shared index is a real leak path requiring automated proof, not review
- Two cost models and two capacity-planning exercises
Choose differently when. Pooled-only is right for a purely commercial product with no regulated customers. Siloed-only is right when every customer is regulated and the price supports it.
Lesson. Make isolation a product tier with a price, not an engineering promise. Then the customer chooses the trade-off, and the architecture stops pretending it can avoid one.
ADR-26 · Configuration, not custom code — but the configuration is engineered
Status: Accepted · Shown on views: 05, 10, 22
What changes when a new customer wants a different KPI, threshold or approval limit?
Context. The requirement lists eleven things each tenant must be able to configure. "Configuration, not custom code" is the right principle and is routinely undermined in practice, because the configuration is stored as untyped rows in a database, edited live through an admin screen, unversioned, unreviewed and impossible to roll back. That is custom code with worse tooling.
Decision. Tenant configuration — KPI packs, hierarchy, agent packs, data sources, policies, approval thresholds, workflows, dashboards, risk models, terminology and prompts — is a set of versioned, schema-checked artefacts in a repository, promoted through the same rings as code. Industry packs for government, banking, energy and conglomerate are the reusable starting points.
How it works on Azure. One branch per tenant in a configuration repository. The pipeline validates against a JSON schema and rejects unknown keys, so a typo fails the build rather than silently disabling a threshold. Azure Deployment Environments applies configuration alongside the image. The tenant console writes through the same pipeline, so a customer's Tuesday change is still versioned and reversible.
| Option | Verdict | Reasoning |
|---|---|---|
| Versioned, schema-checked artefacts in Git | Chosen | Configuration gets the review, diff, rollback and audit that its blast radius deserves. |
| Database rows edited through an admin UI | Rejected | The usual approach, and the one where nobody can say what changed last Tuesday or put it back. |
| Per-tenant code branches | Rejected | Honest about the divergence and fatal to the product economics by tenant five. |
| A no-code rules engine | Deferred | Attractive for the workflow and threshold subset once the schema has stabilised in production. |
What it buys
- A misconfiguration is diffable, attributable and revertible
- Industry packs make each successive tenant cheaper, which is the commercial thesis
- The same promotion path for code and configuration means one release process
What it costs
- Slower than editing a row, and customers will notice
- A schema to maintain, version and migrate as the product evolves
- The tenant console must write through the pipeline, which is more work than writing to a table
Choose differently when. Direct database configuration is fine for genuinely cosmetic settings with no blast radius. Anything that can change what an executive is told, or what may be approved, belongs in the pipeline.
Lesson. If configuration can break production, it deserves the same rigour as code. "It is only config" is how outages are introduced by people who thought they were being careful.
Sovereignty and security
Residency, key custody, and the authority chain from person to row.
ADR-27 · Residency enforced by policy, network and key custody
Status: Accepted · Shown on views: 21, 26, 28
What makes a data-residency claim true rather than merely stated?
Context. Public-sector customers in the target jurisdiction require data residency and sovereign hosting with nationally mandated information-assurance controls. A contractual assurance is not a control: it cannot stop a developer creating a resource in the wrong region, and it cannot stop a compromised service sending data somewhere else. The claim has to be enforced by something that fails closed.
Decision. Residency is enforced at three layers. Azure Policy at the management-group level denies resource creation outside the permitted region. All PaaS services are reachable only through private endpoints, and the single egress path is an Azure Firewall with an FQDN allow-list. The customer holds the encryption key in a managed HSM, so revocation makes the data unreadable.
How it works on Azure. A management group per sovereignty class carries deny-effect policies on location and on public network access. Hub-and-spoke networking with private endpoints and private DNS for every data and AI service; Azure Firewall Premium is the only route out, with an explicit allow-list. Key Vault Managed HSM holds customer-managed keys for the lake, the decision store, the search index and the evidence store. Customer Lockbox governs vendor support access, and PIM removes standing administrative privilege.
| Option | Verdict | Reasoning |
|---|---|---|
| Policy deny, private-only networking, customer-held keys | Chosen | Three independent controls, each of which fails closed, and one of which the customer holds themselves. |
| Contractual and configuration assurance | Rejected | Cheap, common, and unable to survive the question "what stops it happening?". |
| Platform-managed keys | Rejected | Operationally simpler and removes the customer's ability to make revocation meaningful. |
| On-premises or sovereign-partner hosting | Right elsewhere | Sometimes the only acceptable answer, and it forfeits the managed AI services this design is built on. It should be priced as a different product. |
What it buys
- A residency claim that can be demonstrated to an auditor rather than asserted
- Key revocation is a real customer-held control
- A misconfiguration cannot deploy out of region, because the platform refuses
What it costs
- Private networking makes development and diagnosis materially harder
- Customer-managed keys introduce a customer-caused outage mode that must be understood on both sides
- Policy exceptions become a governed process with expiry dates
Choose differently when. Lighter controls are proportionate for a commercial tenant with no residency obligation — and the tier model in ADR-25 is what lets them have that without a second product.
Lesson. A sovereignty claim is only as strong as the control that fails closed when someone tries to break it. Everything else is a sentence in a contract.
ADR-28 · The caller's authority reaches the row; no read-everything identity exists
Status: Accepted · Shown on views: 26, 27
Whose permissions apply when an agent reads a table?
Context. The requirement is explicit that the AI must inherit the user's authorisation context, and contrasts it with the pattern where a privileged identity reads freely and the application filters afterwards. The second pattern is far easier to build and turns every application bug into a data breach, because the only thing between one executive and another's data is a correct WHERE clause in code nobody reviews as a security control.
Decision. The caller's identity is exchanged on behalf of and carried into every downstream read. Row-level security in the semantic model and group filters in the search index are evaluated as the caller. No service identity exists that can read all tenant business data.
How it works on Azure. Entra ID on-behalf-of exchange at the Experience API produces a downstream token with the user's scopes. The tool plane passes it to the semantic model, where row-level security applies, and builds the search filter from the caller's group claims. Managed identities are used only for the platform's own operational stores. Conditional Access, device compliance and MFA gate the original sign-in.
| Option | Verdict | Reasoning |
|---|---|---|
| End-to-end on-behalf-of | Chosen | If the person cannot see it in the source system, nothing here can surface it to them. |
| Privileged read identity with application-side filtering | Rejected | Simpler, faster, cacheable — and every filtering bug becomes a breach. |
| Copy permissions into the platform and evaluate locally | Rejected | A second permissions system to keep in step with the first, which it will not stay in step with. |
| Per-tenant service identity with coarse role mapping | Right elsewhere | Workable where the source system genuinely cannot express per-user permissions — and then that source is limited to aggregates with no drill-down, and the limitation is stated. |
What it buys
- Authorisation correctness is inherited from systems that already got it right
- No privileged identity exists to be stolen
- The security story is one sentence a CISO can check
What it costs
- Caching becomes hard, which ADR-14 addresses at the cost of hit rate
- Scheduled work has no user to act as and needs its own constrained pattern
- A source that cannot express permissions constrains what the platform can offer over it
Choose differently when. A service identity is acceptable where all data is uniformly accessible to all users of the platform. In an executive context spanning HR, finance and procurement, it never is.
Lesson. Inherit authorisation; do not re-implement it. The second implementation is the one that will be wrong, and nobody will notice until it matters.
ADR-29 · Regional model availability is carried as a contracted risk, not an assumption
Status: Accepted · Shown on views: 21, 28
What happens if the models this design assumes are not available in the sovereign region?
Context. The design places inference in country, which is what makes the sovereignty argument work. AI service and model availability varies by Azure region, changes over time, and is outside the programme's control. Quietly assuming availability is the most likely way for this architecture to be invalidated after signature, and it is precisely the kind of assumption that architecture documents bury.
Decision. Regional model and service availability is treated as an explicit, named risk with three pre-agreed fallbacks, verified against the region's current service list before contract and re-verified at each major release. The choice of fallback belongs to the customer and is recorded in the contract rather than decided by the platform team.
How it works on Azure. The tenant registry records the permitted inference region per tenant, and the orchestrator routes model calls accordingly, so a fallback is a configuration change rather than a redeployment. Azure Policy still denies data-at-rest outside the permitted region regardless of where inference runs, which keeps the residency claim intact under fallback two.
| Option | Verdict | Reasoning |
|---|---|---|
| Named risk with three contracted fallbacks | Chosen | The honest position: the platform team cannot control this, so the customer decides the trade-off with the facts in front of them. |
| Assume availability and design around it later | Rejected | The common approach, and it converts a known constraint into a post-signature crisis. |
| Deploy only the models the region has today | Deferred | Fallback one. Correct where the capability gap is acceptable, and it should be a decision rather than a default. |
| Self-host open-weight models in region | Right elsewhere | A genuine answer to the sovereignty constraint, and it trades managed-service economics and quality for control. Worth costing properly if the region's service list stays thin. |
What it buys
- The largest external dependency is visible to the sponsor before commitment rather than after
- Routing per tenant means a fallback does not require re-architecture
- Data residency survives fallback two, because storage location is enforced separately from inference location
What it costs
- A commercial conversation that is easier to postpone than to have
- Fallback two weakens the strongest version of the sovereignty claim and must be described accurately
- Re-verification is recurring work at every major release
Choose differently when. Where the target region's AI service list is mature and stable, this becomes a routine dependency check rather than a headline risk — but it remains a check.
Lesson. Name the dependency that could invalidate your design, and price its alternatives before signature. An architecture document that hides its biggest assumption is marketing.
Operations and assurance
Releasing safely, degrading honestly, and proving it afterwards.
ADR-30 · Evaluation is a release gate, and a prompt is a release artefact
Status: Accepted · Shown on views: 22, 24
What stops a prompt change quietly degrading every tenant's answers?
Context. A prompt change can break accuracy as thoroughly as a code change and is far easier to make casually, often by someone improving a phrase. Model upgrades have the same property and arrive on the vendor's schedule rather than ours. Without a gate, quality regression is discovered by an executive, which is the most expensive possible detection mechanism.
Decision. Prompts are versioned artefacts released with the container image and rolled back with it. Every release runs a golden-question suite, scoring retrieval and generation separately, with a no-regression rule. Tenants may pin a model version and opt in to upgrades on their own schedule. Production failures are mined back into the suite.
How it works on Azure. The prompt registry is versioned configuration. Azure AI Foundry's evaluation harness runs the suite in the pipeline against a staging deployment; retrieval metrics and generation metrics are gated separately so a regression has a stage. Ring deployment sends the change to one pilot tenant before all tenants, and view 23's grounding-rate signal is watched for twenty-four hours afterwards.
| Option | Verdict | Reasoning |
|---|---|---|
| Evaluation as a pipeline gate, prompts as artefacts | Chosen | Quality regression is caught by the pipeline rather than by a chief executive. |
| Manual spot checks before release | Rejected | Works for the first three releases and silently stops working around release ten. |
| Prompts editable in production configuration | Rejected | Fast to fix a phrasing problem, and it makes every answer's provenance unknowable. |
| Vendor benchmark scores as the gate | Rejected | Measures the model on someone else's questions, not the platform on this tenant's. |
What it buys
- A regression has a stage and an owner rather than a shrug
- A tenant's own questions become the acceptance criteria, which is also a good sales artefact
- Model deprecation becomes a scheduled migration instead of an emergency
What it costs
- Evaluation runs cost real money on every release and will be the first target of a cost exercise
- Golden sets rot and must be actively maintained from production failures
- Release cadence slows, which the team will feel before the customer feels the benefit
Choose differently when. Lighter evaluation is proportionate for an internal assistant where a wrong answer costs a few minutes. It is not proportionate anywhere an answer becomes an approval.
Lesson. Anything that changes the output is a release artefact. If it can be edited in production without a gate, it will be, on a Friday.
ADR-31 · Evidence is snapshotted, and the writer cannot delete
Status: Accepted · Shown on views: 06, 11, 30
What does an auditor see when they ask what the approver saw?
Context. Re-running the query that produced a figure gives today's number. Four months later the underlying data has been restated, a mapping has been corrected, or a period has closed — so a decision that was correct at the time looks negligent. Separately, an audit trail that the application can delete from is not an audit trail; it is a log.
Decision. Evidence is a snapshot taken at the moment of the answer and stored immutably, never re-derived. The identity that writes the audit trail has no delete permission, and immutability is enforced by the storage service rather than by application logic. Options that were shown and rejected are retained.
How it works on Azure. Evidence payloads are written to Azure Blob Storage under a time-based immutability policy with legal hold, and referenced from the decision record by identifier. A per-tenant, per-day hash chain covers the record set. The writing identity holds a write-only role assignment; deletion requires a separate, PIM-elevated identity that cannot write. Access events flow to Sentinel, so reading a decision record is itself audited.
| Option | Verdict | Reasoning |
|---|---|---|
| Immutable snapshots, writer cannot delete | Chosen | The record survives both the passage of time and a compromise of the application that wrote it. |
| Re-run the query at audit time | Rejected | The intuitive approach, and it answers a different question from the one being asked. |
| Store the query and the parameters only | Rejected | Compact, and it depends on the data being unchanged, which is exactly what cannot be relied on. |
| External notarisation of the hash chain | Deferred | Worth adding where an auditor must verify without trusting the operator at all; the internal chain plus storage immutability meets the stated requirement today. |
What it buys
- A decision can be judged on what was known at the time, which is the only fair basis
- An application compromise cannot rewrite history
- The evidence pack in view 30 is a mechanical export rather than a reconstruction
What it costs
- Storage grows with every answer, and ten-year retention makes that a real, ongoing cost
- Immutability means mistakes are permanent, including accidentally captured sensitive content
- A separate deletion identity is operational friction, deliberately
Choose differently when. Storing the query alone is acceptable where the underlying data is genuinely immutable — an append-only event log, for instance. Very little enterprise finance data qualifies.
Lesson. An audit answers "what did they know then?", not "what is true now?". Systems that re-query at audit time are answering the wrong question confidently.
ADR-32 · A written degradation contract, and one place the platform fails closed
Status: Accepted · Shown on views: 23, 25
What does an executive get at 07:10 when a dependency is down?
Context. The requirement asks for graceful degradation when an enterprise system is unavailable. That is only real if it is written down per dependency and per capability, agreed with the tenant and exercised — otherwise it resolves to a 500 during the morning brief, which is when the platform's reputation is actually set.
Decision. A degradation matrix states, per dependency and per capability, what still works, what degrades and what stops. Every degraded state names itself in the answer. The platform fails open with a stated caveat everywhere except one case: if the decision cannot be recorded, it refuses to act rather than acting unrecorded.
How it works on Azure. Health of each dependency is evaluated by the tool plane before composition, and the resulting capability state is attached to the answer. Cached KPI values are served up to twenty-four hours with the as-of date shown, then withheld. Approvals queue in Service Bus when the execution plane is unavailable and are re-authorised on return. Degradation states are exercised in a quarterly game day per tenant tier.
| Option | Verdict | Reasoning |
|---|---|---|
| Written matrix, fail open with a caveat, fail closed on the record | Chosen | Every state is a decision someone made, and the one exception is where an unrecorded action would be worse than no action. |
| Fail closed everywhere | Rejected | Safest and it makes a single stale nightly feed silence the entire morning brief. |
| Fail open silently | Rejected | The worst option: a confidently partial answer is more dangerous than an error, because nothing signals the gap. |
| Best-effort, undocumented | Rejected | The default when nobody writes the matrix, and it means the behaviour is discovered during an incident. |
What it buys
- Degradation is a designed behaviour with an owner rather than an emergent one
- The tenant knows in advance what an outage looks like, which is a contractual conversation had calmly
- The one fail-closed case is defensible precisely because it is the only one
What it costs
- Every capability needs a degraded path built and tested, which is real engineering
- Game days cost time per tenant tier
- More states in the user interface to design and explain
Choose differently when. Fail closed more broadly where a partial answer carries regulatory consequence — clinical or safety-critical contexts, for instance, where a caveat is not sufficient protection.
Lesson. Decide where you fail open and where you fail closed, write it down, and test it. A system without a degradation contract has one anyway; it just has not been reviewed.
The Azure services used, in one table
Every Azure service on these thirty views, what it is, and what it is doing here. The last column names what was considered instead, so the table doubles as a shortlist for anyone adapting this design to a different estate.
| Service | What it is | What it does here | Considered instead |
|---|---|---|---|
| Azure Front Door Premium | Global anycast edge with WAF and DDoS protection | TLS termination and the only public ingress; reaches the platform over Private Link | Application Gateway, Cloudflare, Akamai |
| Azure API Management | API gateway with policy, quota and contract enforcement | Per-tenant rate and token quotas, contract validation, and the tool plane's front door | Container Apps ingress with custom middleware |
| Azure Container Apps | Managed serverless container platform | Runs the experience API, orchestrator host, decision, config and governance services | AKS, App Service |
| Azure Functions (Durable) | Serverless compute with durable orchestration | Long-running investigations with checkpoints, resumable across a host restart | Service Bus with a hand-built state machine |
| Azure Bot Service | Managed bot hosting with channel connectors | Delivers the morning brief and approvals as Adaptive Cards in Teams | Direct Microsoft Graph integration, Copilot Studio |
| Azure AI Foundry | The AI platform: agent runtime, model catalogue, evaluation and tracing | Agent threads, tool invocation, OpenTelemetry traces and the evaluation harness used as a release gate | LangGraph, AutoGen, a hand-rolled loop |
| Azure OpenAI | Managed OpenAI models with enterprise networking and residency | A reasoning model for planning and composition, a small model for routing, plus embeddings | Open-weight models on managed compute |
| Azure AI Search | Search service with vector, keyword and semantic ranking | Enterprise retrieval with hybrid queries and index-side security trimming | pgvector on PostgreSQL, Elasticsearch |
| Azure AI Content Safety | Safety services including prompt shields and groundedness detection | Independent verification that every claim is supported by retrieved evidence | A second LLM as judge, open-source guardrails |
| Azure AI Document Intelligence | Document layout and structure extraction | Turns PDFs, contracts and board papers into structured, chunkable content | Open-source PDF parsing |
| Microsoft Fabric | Unified analytics platform: OneLake, lakehouse, warehouse, pipelines, real-time | The analytical foundation, with shortcuts to the customer's existing lake instead of copies | Synapse + ADLS, Azure Databricks |
| Power BI semantic model | Tabular model with DAX measures and row-level security | The single definition of every KPI, shared with the customer's own reporting | dbt metrics, a bespoke metric service |
| Azure Cosmos DB (Gremlin) | Distributed database with a property-graph API | Canonical entity graph for multi-hop questions and signal propagation | Recursive SQL, a dedicated graph product |
| Azure SQL Database | Managed relational database | The decision, evidence, action and outcome record, with row-level security and point-in-time restore | Cosmos DB NoSQL, PostgreSQL Flexible Server |
| Azure Machine Learning | Model registry, training and managed online endpoints | Forecasting, anomaly detection and risk scoring as versioned, served artefacts | Fabric Data Science notebooks in-line |
| Azure Cache for Redis | Managed in-memory cache | Conversation state and a semantic cache keyed on the caller's permission fingerprint | State in the decision store, no cache |
| Azure Service Bus | Enterprise message broker with peek-lock and dead-lettering | The approved-action queue between the decision service and the execution plane | Storage queues, a database outbox |
| Azure Logic Apps Standard | Workflow engine with a large enterprise connector library | The execution plane: the only component holding write credentials, one identity per target system | Custom connector services |
| Azure Blob Storage | Object storage with immutability policies and legal hold | Immutable evidence snapshots and the audit archive, written by an identity that cannot delete | Evidence in the database, a third-party WORM archive |
| Microsoft Entra ID | Identity platform with OAuth, OBO, Conditional Access and PIM | Sign-in, and the on-behalf-of chain that carries the caller's authority to the row | A privileged service principal with application-side filtering |
| Azure Key Vault Managed HSM | FIPS-validated hardware security module as a service | Customer-held encryption keys per tenant, making revocation a real control | Standard Key Vault, platform-managed keys |
| Microsoft Purview | Data governance: catalogue, classification, lineage | Sensitivity labels captured at ingestion, and source-to-measure lineage in the evidence pack | Manual cataloguing, pipeline-inferred lineage |
| Azure Monitor and Application Insights | Metrics, logs and distributed tracing | End-to-end turn traces following OpenTelemetry GenAI conventions, plus cost per interaction | A third-party APM |
| Microsoft Sentinel | Cloud-native SIEM and SOAR | Security analytics over access events, including reads of decision records | The customer's existing SIEM with forwarding |
| Microsoft Defender for Cloud | Cloud security posture management and workload protection | Posture assessment across the tenant subscriptions and the control plane | Third-party CSPM |
| Azure Firewall Premium | Managed network firewall with TLS inspection and FQDN filtering | The single controlled egress path; everything not on the allow-list is denied | NSG rules alone, a network virtual appliance |
| Azure Policy and management groups | Governance controls applied above the subscription | Deny resource creation outside the permitted region and deny public network access | Review-based governance, Terraform Sentinel |
| Azure Bicep and Deployment Environments | Declarative infrastructure as code, and governed environment provisioning | Tenant onboarding, with the isolation tier as a deployment parameter | Terraform, manual provisioning |