A reporting database on a burstable instance class has run well for months. A new nightly job starts at 22:00. By 04:00 queries that took 200 ms take 9 seconds. By 07:00 everything is normal again. CPU reads 100% through the window, and the instance's average utilisation for the day is 20%. Explain it.

The trigger

Burstable instance families and some older volume types do not give you their headline performance continuously. They accrue credits at a baseline rate and spend them whenever the workload exceeds that baseline. While the balance is positive, the instance behaves like a much larger one. When it reaches zero, performance is capped at the baseline - for the small burstable sizes that is a fraction of a single vCPU - and the drop is a cliff, not a slope.

The nightly job ran above baseline for several hours, drained the balance somewhere around 04:00 minus the accrual, and from that moment the database was running at a few tens of percent of one core. A 200 ms query became a 9-second query. Once the job finished, credits accrued again and by 07:00 the instance was back to its apparent normal.

Why it looked healthy for months

Because the workload had never exceeded baseline for long enough to empty the bucket. The design is honest about this and the metric is published - the credit balance - but it is not on any default dashboard, and every signal teams do watch is actively misleading:

• Average CPU utilisation (20%) is the mean over the day and says nothing about a six-hour window.
• CPU at 100% during the window looks like a sizing problem, which invites a query-tuning investigation rather than an instance-class one.
• Latency recovered on its own, which makes it look transient.

The same shape in storage

Older general-purpose volumes work the same way: throughput and IOPS are provisioned per gibibyte - 3 IOPS per GiB for AWS's gp2, with a burst bucket that lets smaller volumes reach 3,000 IOPS - and the bucket drains under sustained load. A 100 GiB gp2 volume delivers 300 IOPS once its burst is gone, which is the storage version of the same 04:00 cliff.

The fix

1. Move to a fixed-performance class for anything whose latency matters at predictable times - a non-burstable instance family, and gp3-style volumes where IOPS and throughput are provisioned explicitly rather than earned.
2. If the burst model genuinely fits, alert on credit balance trending to zero, which is the only predictive signal available. Everything else is after the fact.
3. Consider the surplus-charging mode where the provider offers it (AWS's T-unlimited), which converts the cliff into a cost increase - a much better failure mode, and one that needs a budget alert so it does not become a surprise invoice.
4. Move the nightly job off the serving instance, which is the answer most teams should reach regardless of instance class.

When this is the wrong thing to blame

For development and test environments, internal tools, and services whose load is genuinely spiky and small, burstable classes are substantially cheaper and the credit model is a fair trade. The mistake is not choosing them; it is choosing them for a workload with a sustained nightly peak and then not monitoring the one metric that predicts the failure.