Embedding Pipeline Service · View 18 of 22 · Operations
The signal that does two jobs
- Oldest-unembedded age per lane is the freshness SLI and the scaling trigger. Queue depth cannot distinguish a large backlog from a stuck one; an age can.
- The same shape repeats per stage — oldest unprepared, oldest unmigrated — so a delay can be attributed to a stage rather than to the pipeline.
- Retrieval p99 is the only latency SLI, because it is the only number a user experiences.
Cost is a monitored quantity
- Chunk reuse rate and vector cache hit rate are tracked as operational signals, not reports: a drop in either is a cost incident before it is a budget variance.
- GPU batch efficiency makes the freshness-versus-cost trade measurable rather than asserted.
- Migration burn against estimate is watched during a migration, so an overrun is caught at day three rather than at the invoice.
Quality is not an afterthought
- Vector distribution drift and query distribution drift are monitored separately, because they look identical and have different fixes.
- Index recall against a brute-force sample distinguishes an ANN parameter problem from a model problem.
- Results-withheld counts sit under correctness, not compliance: a spike usually means the permission authority is unwell, not that policy tightened.
