Embedding Pipeline Service  ·  View 05 of 22  ·  People and journeys

Journey — Ship a New Corpus

A product engineer onboarding retrieval over a new content type, and the day-one problem nobody has an answer to.

Editable source SVG draw.io All views
Product engineer owns one product surface Goal — Get retrieval over my corpus into production without owning GPUs, a chunker or an index Trigger — A product commitment to ship search over a new content type this quarter Done when — Retrieval live, quality measured against a query set I wrote, and a cost figure I can defend 1 · Register self-service 2 · Backfill bulk lane 3 · Measure ◆ moment of truth 4 · Launch ◆ moment of truth 5 · Own it What they do Declares the corpus Picks a freshness tier Waits on the backfill Writes 80 eval queries Switches traffic on Watches the dashboard What the platform does Contract assigned Index provisioned Bulk lane, preemptible Progress + ETA Scores recall@10 Alias published Cost + freshness showback How it feels Confident Uneasy Blocked Where it hurts No ETA, unclear if it is stuck Nobody has labelled relevance Bill arrives without a cause What answers it Chunker chosen by corpus type Oldest-unembedded age as the ETA Harness scores an unlabelled set Recall gate before the alias flips Reuse rate + cache hit as the two cost ratios Journey — Ship a New Corpus The trough is measurement: nobody has ground truth on day one. The platform's answer is a frozen query set with reusable judgements, not a label budget. v 1.0 · owner Data & AI Platform Architecture · date 2026-10

The trough, and what answers it

  • Measurement is the dip: on day one nobody has labelled relevance data for the new corpus, and a launch gate that needs labels will be skipped.
  • The answer is a frozen query set the team writes themselves, scored for recall@10 and MRR, with judgements that are reused on every later contract change rather than re-bought.
  • The second pain — an opaque backfill — is answered by oldest-unembedded age as the ETA, which distinguishes a long backlog from a stuck one.

Self-service boundary

  • A team declares a corpus, its change-feed mechanism, its permission model and a freshness tier. The platform assigns the contract, the chunker and the index.
  • The team does not choose a chunker, a model or an index topology. Those are platform concerns precisely because they are contract elements.
  • Cost comes back as two ratios the team can act on — chunk reuse rate and cache hit rate — rather than a bill with no cause.

Assumptions

  • 14 consuming teams; a new corpus is onboarded roughly monthly.
  • A backfill of a typical new corpus runs in the bulk lane on preemptible GPU capacity, with no freshness SLO and a published completion estimate.
◀  Journey — Find What I Just Wrote All views Journey — Upgrade the Embedding Model  ▶
Solution Architecture v1.0 · open source, self-hosted on Kubernetes · Data & AI Platform Architecture · 2026-10 · 22 views · 17 architecture decision records05-journey-ship-a-corpus