advanced 2 min answer

A team says 200 ms is the minimum achievable latency for an operation. How would you check?

first-principleslatencyassumptionsphysicsanalysis
Show the full answer Hide the answer

What is being tested

Whether you can decompose an asserted limit into what is genuinely fixed and what is convention.

The method

Calculate the floor from what is actually fixed.

  • Network round trips. Speed of light in fibre gives roughly 1 ms per 100 km each way. Within a region: under 1 ms. Cross-continent: 60–80 ms. Intercontinental: 150 ms or more. Count the round trips in the path and multiply.
  • Serialisation and transfer. Payload size divided by bandwidth. Usually small; occasionally not.
  • The actual computation. How many rows read, how much work per row.
  • Storage access. Memory is nanoseconds; SSD is tens of microseconds; a network hop to a database is the round trip above.

Sum them. If the floor is 40 ms and the observed figure is 200 ms, 160 ms is architecture, not physics, and the question becomes where it is going.

Then find the difference

A trace answers this faster than analysis: where is the wall-clock time spent? Almost always one of:

  • Sequential calls that could be parallel. Six sequential 30 ms calls is 180 ms; the same six in parallel is 30 ms.
  • Queueing, because utilisation is high. Waiting time rises non-linearly near saturation — 4x the service time at 80% utilisation.
  • Connection setup, paying handshake and TLS repeatedly because connections are not pooled.
  • Work that could have been precomputed.
  • A timer. Consistent round-numbered latency is a timer, not work — 40 ms, 100 ms and 200 ms all suggest a configured or protocol interval rather than computation.

The assumption to challenge explicitly

"Must this be computed at request time?" If the answer can be precomputed, the latency floor is a read from a store, which is an entirely different number. Most latency assertions rest on an unexamined assumption that the work must happen now.

The general method

  1. State the goal in measurable terms.
  2. List every assumption in the current approach.
  3. Ask of each: is this a law, a constraint, or a convention? Most are conventions.
  4. Reason up from what is genuinely fixed.

Step three is where the value is. "We need a database" is a convention; "we need durable state with these access patterns" is closer to the requirement and admits answers a database does not.

The caution

First-principles reasoning tells you what is possible, not what is worth building. Use it to understand constraints and evaluate options — not as a licence to rebuild solved problems.