Storage Tiering Service  ·  View 11 of 31  ·  3 · Structure

Anatomy of a Read

Everything between a file service asking for an object and the first byte, and why every shortcut on this path is safe.

Editable source SVG draw.io All views
Ask File service key · reader class Resolve Resolver cache TTL 60 s Catalogue replica lag cut-off 5 s Catalogue primary only on not-found Route Recall class from placement Read Read router pack offset → range Job handle ETA · within 3 s Bytes Ceph tier RGW GET · ranged delayed · scheduled Anatomy of a Read — Resolve, Route, Read External / third party Interface / broker Data store Decision point Application we own failure / alternate A source is released 24 h after commit: longer than the cache TTL, the replica lag and the longest stream. A stale answer still points at bytes. v 1.0 · owner Storage Platform Architecture · date 2026-09

Decisions

  • Resolution tries a 60-second in-process cache, then a replica whose lag is under 5 seconds, then the primary only when the row is not found. Only brand-new uploads reach the primary.
  • That order is safe because the source of a movement is not released until 24 hours after commit. Any answer that is 60 seconds, 5 seconds or even an hour stale still names a location that holds the bytes (ADR-03).
  • Packed objects are read by range from the pack; the router turns a placement's pack offset and length into a ranged GET. A cold pack read never reconstructs the whole pack.

Numbers

  • Resolution p50 2 ms, p99 8 ms. First byte p99 45 ms hot, 120 ms warm, 900 ms cold.
  • About 0.7% of reads resolve to delayed or scheduled. They get a job handle within the 3-second synchronous ceiling and are never held open.

Risk

  • A GET that returns 404 at a committed location is drift, not a miss. The router retries once with an uncached resolve and then fails loudly and reports it, because silently probing other tiers would put the tiering plane's job on the read path.