A product manager asks to tighten the API latency target from p99 300 ms to p99 100 ms. What should the architect do before agreeing?

What is being tested

Whether you treat a quality attribute as a purchase with a price rather than as a preference.

The reasoning

A tighter target is not a tuning exercise; past a point it is a structural change. Going from 300 ms to 100 ms at p99 typically means the request can no longer afford:

• A cross-region round trip. If the database primary is in another region, you have already spent most of the budget on physics. The fix is regional read replicas or regional writes, which imports replication lag or conflict resolution.
• A cold cache. 100 ms p99 usually implies a very high cache hit rate, which implies pre-warming, longer TTLs, or precomputation — all of which import staleness.
• Chained synchronous calls. Three sequential hops at 30 ms each leaves nothing for the work. The fix is parallelising, denormalising, or precomputing a joined view.
• Garbage collection pauses and cold starts. At p99 these stop being noise and become the dominant term.

Each of those fixes has a name and a bill. The architect's job is to say: this costs roughly this much money and introduces this much staleness — is the latency worth it? That question can only be answered by the person who owns the business outcome, and they can only answer it if the price is stated.

The second question worth asking

Measured where? p99 at the service excludes DNS, TLS, the client's network and the render. A user-perceived improvement may be far cheaper to buy at the edge — caching, connection reuse, regional termination — than in the service. Many latency requirements are satisfied without touching the backend at all, and finding that out is worth an hour.

The third question

Why 100? If the real requirement is "the page should not feel slow", the honest target may be about consistency of latency rather than its absolute value. A p99 of 250 ms with a tight distribution often feels better than a p99 of 100 ms with a long tail of 3-second outliers.

Common weak answers

Agreeing immediately, which trains the organisation to believe NFRs are free. Refusing, which is not the architect's call to make. Scaling horizontally, which reduces queueing delay but does nothing for a latency floor set by round trips and serialisation.