A team is asked to "make the system faster" for a delivery platform. What are they actually being asked, and why does the distinction change the work?
Show the full answer Hide the answer
The two things "faster" can mean
Latency is how long one operation takes. Throughput is how many operations complete per unit time. They are related and frequently traded against each other, and optimising one can worsen the other.
Batching improves throughput and worsens latency. Adding concurrency improves throughput and worsens latency past the efficient point. Caching improves both. Compression improves throughput on a constrained network and worsens latency by adding CPU.
Which one the business means
For a delivery platform the answer differs by path and the team must ask:
- The customer-facing app is a latency problem. Nobody cares how many page loads the system can serve per second; they care that theirs completes.
- The dispatch and assignment engine is a latency problem with a hard deadline — a decision that arrives after the courier has moved is worthless.
- The location ingestion pipeline is a throughput problem. Individual message latency is irrelevant; keeping up with the fleet is everything.
- Overnight analytics and route optimisation is a throughput problem with a deadline measured in hours.
Applying a latency optimisation to a throughput problem, or the reverse, is the most common wasted quarter in performance work.
The third thing that is often meant
Predictability. "Faster" frequently means "not occasionally terrible" — the p99 rather than the mean. That is a different investigation, with different causes (queueing, cache misses, garbage collection, fan-out tail amplification) and different fixes, and it is worth confirming before starting.
The question to ask
"What decision or experience changes if this is faster, and by how much?" A latency improvement that no user perceives has no value; a throughput improvement that removes a nightly deadline breach has a lot. Getting this answer before starting is what separates performance work that matters from performance work that is merely satisfying.