Enterprise Generative Search — Azure and Open Source  ·  View 30 of 41  ·  Operations

Deployment Architecture

Two regions, four node pools, and a failover that does not depend on replicating a search index.

Editable source SVG draw.io All views
Azure — West Europe (primary region) Hub VNet — edge and shared services Front Door and WAF TLS, bot rules API Management quota, keys, versions Azure Firewall egress allowlist Key Vault workload identity Bastion no public admin AKS spoke — application node pools, 3 availability zones Answer plane 38 pods, D8s v5 Retrieval plane 26 pods, D8s v5 Ingest plane autoscaled 4 to 60 Platform plane gateway, policy, traces AKS spoke — GPU node pools Embed and rerank 4 x A10, always on Verifier and small LLM 2 x A10 Batch embedding 6 x A10 spot, queue-driven Data services — private endpoints only Azure AI Search S3, 12 partitions, 3 replicas PostgreSQL Flexible zone-redundant HA Cache for Redis premium, zone-redundant ADLS Gen2 ZRS, immutable raw Event Hubs and Service Bus zone-redundant Azure — North Europe (paired region, warm) Standby capacity AKS minimal scales on failover AI Search replica index indexer writes both PostgreSQL read replica promotable Recovery targets RTO 30 minutes search first, then answers RPO 5 minutes ledger and provenance only Azure OpenAI two regions, one gateway Microsoft Entra ID global Databricks governed SQL warehouse Content sources read-only TLS 1.3 mTLS query second index build async replication Deployment Architecture — Where It Runs and What Survives a Failure Security / platform Interface / broker Application we own Data store Queue / topic Decision point External / third party synchronous batch event / async The secondary is not a copy of the primary: the indexer builds both, so failover does not depend on replicating a search index that Azure AI Search does not replicate for us. v 1.0 · owner Data and AI Global Practice

Decisions

  • The indexer writes both regions. Azure AI Search does not replicate an index across regions for us, so the secondary is built rather than copied — which also means the failover path is exercised continuously by normal ingestion.
  • GPU pools are separated by purpose: always-on for online embedding and reranking, and preemptible spot for batch embedding. A reindex must never take capacity from the query path.
  • Every data service is reachable only over a private endpoint, and the only inbound path to the platform is Front Door.

Numbers

  • RTO 30 minutes and RPO 5 minutes, both applying to zone 3 data in view 17. Search comes back first, generative answers second.
  • Three availability zones for application pods and all zone-redundant data services; a zone loss is a capacity event, not an outage.
  • 12 GPU nodes steady state, of which 6 are spot and interruptible without user-visible effect.

Risks

  • The warm secondary carries a standing cost for an event that may never happen. It was justified by the RTO, and the alternative — rebuild on demand, 14 hours — was rejected against it explicitly.
  • A regional Azure OpenAI capacity event is not solved by our failover. The gateway holds deployments in two regions and will route across them, which is a mitigation rather than a guarantee.
◀  Degradation Ladder All views CI/CD and Index Migration  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 41 views · Microsoft Azure and open source30-deployment-architecture