Multimodal RAG Platform · View 14 of 16
RAGOps Lifecycle
The improvement loop, drawn as a loop. A RAG system is never finished at launch.
Copy
PNG
PDF
⋯
Editable source
SVG
draw.io
All views
Observe
traces, thumbs, escalations
Observe...
Diagnose
retrieval or generation
Diagnose...
Curate
add to golden set
Curate...
Change
chunking, prompt, model
Change...
Evaluate
offline then shadow
Evaluate...
Release
canary, gated
Release...
RAGOps
RAGOps
failure signals
failure signals
labelled cases
labelled cases
hypothesis
hypothesis
candidate build
candidate build
eval verdict
eval verdict
new baseline
new baseline
RAGOps — The Improvement Loop
RAGOps — The Improvement Loop
Diagnose first splits retrieval failure from generation failure. The two have different fixes.
Diagnose first splits retrieval failure from generation failure. The two have different fixes.
v 1.0 · owner AI Engineering
v 1.0 · owner AI Engineering
Text is not SVG - cannot display
The loop
Observe traces, thumbs and escalations
Diagnose, curate, change, evaluate, release
Each release becomes the next baseline
Diagnose first
Split retrieval failure from generation failure before changing anything
Retrieval failure: the right chunk was never returned
Generation failure: it was returned and the answer still went wrong
The two have entirely different fixes
Cadence
Weekly triage of the failure queue
Golden set grows from real failures, never synthetic ones
No change ships without an offline eval verdict
