Storage Tiering Service  ·  View 21 of 31  ·  5 · Runtime

Retrieval Storm

Four simultaneous demands on the same sixteen tape drives, and how each is admitted, planned and bounded.

Editable source SVG draw.io All views
Arrives Admitted by Planned as Draws on Staged for Ends Interactive reads 0.7% of reads non-immediate Job handle ≤ 3 s Single object 2 drives reserved 48 h Promotion by policy eDiscovery · legal 0.7 M archived objects Legal authority never refused Packs by cartridge Up to 8 drives 7 days Unread bytes charged Tenant migration export 42 M objects Tenant budget hard ceiling Queued behind legal Remaining drives 48 h Paused at ceiling resumable ML training scan Whole corpus non-promoting Immediate tiers only Archive excluded Ceph · 2 Gbps cap No staging No placement change Retrieval Storm — Four Demands on Sixteen Tape Drives The storm degrades to a longer published ETA, never to a larger bill. Drive reservations are ladder configuration, not code. v 1.0 · owner Storage Platform Architecture · date 2026-09

Decisions

  • Drives are reserved by demand class: two always kept for interactive recalls, up to eight for legal authority, the rest shared. An eDiscovery sweep cannot starve a user who opened an archived file.
  • The ML training scan is admitted to immediate tiers only and capped at 2 Gbps. Tape is excluded unless a job is explicitly authorised, because training on archived data is a decision, not a side effect.
  • A tenant export that hits its budget ceiling pauses with its staged bytes kept for their TTL. The platform does not throw away retrievals it has already paid for to enforce a budget.

Numbers

  • Bulk recall planning for up to 50 million objects in one job. Queue depth and the current estimate per tier are published every minute.
  • Two libraries with eight LTO-10 drives each; about 8 PB of archive bytes, two copies, roughly 550 cartridges.

The principle

  • A storm degrades to a longer published estimate. It never degrades to an unbounded bill or a hung request.