Profiling and Optimisation

Definition

Optimisation is the work of reducing resource consumption for the same output. It has a method, and the method's first two steps are the ones most often skipped.

The method

1. Establish the objective. "p99 under 300 ms" or "40% less compute cost". Without a target, optimisation has no stopping condition and consumes unbounded engineering time.

2. Measure. Profile under production-like load and data. Intuition about where time goes is reliably wrong, and the classic outcome of skipping this is a week spent on something that was 2% of the total.

3. Optimise the dominant term. Amdahl's law bounds the payoff: making a component twice as fast when it is 10% of the total improves the whole by 5%. Find the widest frame, not the ugliest code.

4. Measure again. Confirm the improvement is real and that nothing else regressed. Optimisations frequently move cost rather than removing it.

5. Stop when the objective is met. Further optimisation has a cost — complexity, readability, maintainability — and no benefit once the target is achieved.

The hierarchy of effectiveness

1. Do it less often. Caching, memoisation, incremental processing over full recomputation. The largest wins by a wide margin.
2. Do less work. A better algorithm, fewer rows, a smaller payload.
3. Do it in parallel, where the work is divisible.
4. Do it faster. Micro-optimisation. Last, smallest, and where the most effort is usually spent.

Most significant performance improvements come from the first two, and they are architectural rather than local.

What to watch for

• Optimising the average when the tail is the problem, or the reverse.
• Optimisation that trades readability for a 3% gain in code that is not on the hot path.
• Improvements that do not survive contact with production, because the profile was taken with different data volumes or a warm cache.
• Local optimisation shifting the bottleneck without improving the system — genuine progress requires re-identifying the constraint each round.

The cost dimension

At scale, optimisation is a cost exercise as much as a latency one. Continuous profiling makes "which function is responsible for 8% of our compute bill" a question with a direct financial answer, and that framing frequently unlocks engineering time that a latency argument cannot.

Interview question

"A colleague spent a week optimising a function and the endpoint is 2% faster. What went wrong in the approach?"