Multimodal RAG Platform · View 13 of 16

Observability & Evaluation Matrix

Six signal types across five stages. A matrix rather than a flow, because it makes the gaps visible.

Editable source SVG draw.io All views

Beyond service health

Retrieval quality — recall@k and nDCG from logged hit ranks
Answer quality — groundedness, citation coverage, refusal rate
Cost — tokens and dollars per query and per tenant

Every row ends in Act

Low recall triggers re-chunking or re-embedding
Low groundedness triggers a prompt or model change
Cost drift triggers routing or cache tuning

Safety signals

Guard verdicts emitted on every request
Injection and PII rates tracked as first-class metrics
Write-once audit log feeds the SIEM