Enterprise Generative Search — Azure and Open Source  ·  View 24 of 41  ·  Runtime

Hybrid Retrieval and Fusion

Three hundred candidates down to eight passages, and what each stage is allowed to remove.

Editable source SVG draw.io All views
Prepare Plan and filters ACL, source, freshness Query variants up to 4 Query embedding BGE-M3, cached Generate candidates Lexical BM25 top 100 Dense vector top 100 Learned sparse top 60 Graph expansion top 40 Fuse Reciprocal rank fusion k = 60 Source authority weights policy, not tuning Near-duplicate collapse SimHash on passages Rerank Cross-encoder top 50 scored Score floor 0.32, tuned on golden set Freshness adjustment class-aware, bounded Diversify Maximal marginal relevance lambda 0.7 Two passages per document hard cap Sub-query coverage check each hop represented Select evidence Token budget fit 8 passages, 6k tokens Evidence ids minted chunk plus version Retrieval reason recorded for the trace re-plan dropped Hybrid Retrieval and Fusion — From Four Candidate Sets to Eight Passages Application we own Data store Decision point Security / platform failure / alternate batch Vector-only retrieval failed the golden set on part numbers and policy identifiers, which is why the lexical arm is not an option in the configuration. v 1.0 · owner Data and AI Global Practice

Decisions

  • Reciprocal rank fusion with k=60 rather than score normalisation. Scores from BM25, a dense index and a graph walk are not on a common scale, and pretending they are is the commonest fusion mistake.
  • Reranking depth is 50, not 200. Above 50 the measured nDCG gain on the golden set was inside the noise band and the latency was not.
  • Diversity is applied after reranking, not before. Removing near-duplicates first would discard the strongest passage when a document is genuinely the best source.

Numbers

  • Recall@50: lexical only 0.78, dense only 0.71, hybrid fused 0.92, hybrid plus cross-encoder nDCG@10 0.68 against 0.61 for fusion alone.
  • Retrieval budget 450 ms P95 for all four backends in parallel; reranking 110 ms for 50 candidates on an A10.
  • Two-passage cap per document; median evidence set spans 5 distinct documents.

Assumptions and risks

  • The 0.32 score floor is tuned on the golden set and re-tuned when the reranker changes. A floor that is too high abstains on answerable questions; too low and the model is handed noise.
  • Fusion weights per source are a policy decision recorded in the source register, not a tuning knob an engineer changes at will.
◀  The Bounded Agentic Loop All views Context Assembly  ▶
Solution Architecture v1.0 · Data & AI Global Practice · 2026-09 · 41 views · Microsoft Azure and open source24-hybrid-retrieval-and-fusion