A platform's log volume has grown until log storage is one of its largest infrastructure costs. What should change, and what should not?

What drives the cost

Not storage of the raw bytes, which is cheap. Indexing is what costs: the index is often larger than the data, it must be held in memory or fast storage to be queryable, and it is built at ingestion time regardless of whether anyone ever searches it.

So the highest-leverage change is not deleting logs — it is deciding what to index.

The changes that work

• Index a chosen subset of fields, store the rest. Searching unindexed data is slower and vastly cheaper, and for data queried once a quarter that is the correct trade.
• Tier by age. Recent logs fully indexed on fast storage; older logs unindexed on object storage; oldest deleted or archived. Query patterns follow recency very strongly, so this is close to free in usefulness terms.
• Set retention per log class, not globally. A debug log and an audit record have nothing in common and giving them one policy over-retains one and under-retains the other.
• Sample high-volume, low-value logs at the source — successful health checks, routine request logs for endpoints with a metric already covering them.
• Move what is really a metric out of logs. Counting log lines to derive a rate is enormously more expensive than emitting a counter, and it is a very common pattern.
• Attribute cost per team or service and make it visible. This changes behaviour more than any technical control, because log volume is created by developers who currently see no cost signal.

What should not change

• Do not reduce logging on the critical debugging path. The logs that matter are the ones present during an incident, and cutting them saves money in exchange for longer outages — a trade that is almost never worth it and is easy to make accidentally when cutting by volume.
• Do not delete audit and compliance logs to save cost. They have a different purpose and usually a legal retention requirement.
• Do not aggressively cut before measuring what is queried. Cutting by guesswork removes what someone needed and keeps what nobody wanted.

The structural improvement

Structured logging with consistent field names, which makes selective indexing possible at all. Free-text logs cannot be selectively indexed usefully, so the cost decision reduces to all-or-nothing — and that constraint, not the volume, is what usually makes log costs unmanageable.