A service is slow and CPU utilisation is 4%. What do you profile and what do you expect to find?

What is being tested

Whether you choose the right kind of profiler. A CPU profile of a service that is not using CPU shows nothing useful, and this is the most common profiling mistake.

What to profile

Wall-clock (or "off-CPU") profiling, which includes time spent waiting. The service is slow while idle, so the time is going somewhere that CPU sampling does not observe.

If the tooling supports it, blocking and lock profiling specifically, which attributes time to where threads are waiting on locks, monitors and I/O.

What to expect to find

In rough order of likelihood:

• Waiting on a downstream call. A slow dependency, or a call that could have been made in parallel and is sequential.
• Waiting on a connection pool. The pool is exhausted; threads queue for a connection. CPU is idle because everything is blocked. This is extremely common and presents exactly like this.
• Waiting on a database query. The service is idle; the database is doing the work — or the query is waiting on a lock held by another transaction.
• Lock contention on a shared resource inside the application — a synchronised cache, a singleton, a logging bottleneck.
• Garbage collection pauses, which appear as stalls rather than as CPU load in some collectors and which are best diagnosed with allocation profiling: high allocation rate drives collection frequency, which drives tail latency.
• Thread pool starvation. All workers blocked on something, so new requests queue with no CPU being consumed at all.

The reasoning to state

Low CPU with high latency means the service is blocked, not busy. So the question becomes: blocked on what? Little's Law is the frame — concurrency equals arrival rate times latency, so if latency has risen and traffic has not, concurrency has risen, and something is holding it.

Check pool saturation metrics before profiling if you have them. Connection pool waits and thread pool queue depth frequently answer the question in ten seconds without any profiling at all.

The general profiling discipline

• Profile in production, or under production-like load and data. Development profiles find development problems.
• Profile before optimising. Intuition about where time goes is reliably wrong, and the classic outcome is a week spent on something that was 2% of the total.
• Compare profiles — before and after, or a fast instance against a slow one. The difference is more informative than either alone.
• Look for the surprising width, not the deepest stack. A flame graph's value is the unexpectedly wide frame.

What a strong answer adds

Continuous profiling in production, so you can compare a profile from during the incident with one from last week — and so that "which code path is responsible for 8% of our compute bill" becomes a question with a direct financial answer, which frequently unlocks engineering time that a latency argument cannot.