A platform moves from free-text logs to structured logs. What becomes possible, and what discipline must accompany it to avoid making things worse?
Show the full answer Hide the answer
What becomes possible
Querying by field rather than by regular expression. "All requests for tenant X with latency above 2 seconds in the last hour" becomes a filter rather than a text search that may or may not match.
Aggregation. Counting, grouping and percentiles over log fields, which is impossible when the value is embedded in a sentence.
Correlation. A request id as a field links records across services reliably; as free text it links them approximately.
Automatic enrichment. Service, version, region, deployment and tenant attached uniformly by the logging library rather than remembered by each developer.
Machine-readable alerting, on a field's value rather than on a substring appearing.
The discipline required
1. A field naming convention, enforced. If one service logs user_id, another userId and a third
uid, cross-service queries are impossible and the main benefit is lost. This needs a shared library and a
lint rule, not a wiki page.
2. Bounded cardinality in fields intended for aggregation. A field containing a unique value per request is fine for filtering and useless for grouping, and if the log platform indexes it, the cost is severe. The discipline is to know which fields are dimensions and which are payload.
3. Payload size control. The characteristic failure of structured logging is attaching a large context object to every line, so volume multiplies several times over for information nobody reads. Log the fields that will be queried; put the rest behind a debug level.
4. No sensitive data in fields. Structured logs are searchable and widely accessible, which makes an accidentally logged token or personal identifier far more exposed than the same value buried in free text. Redaction belongs in the logging library, applied by default.
5. Schema evolution treated as an interface change. Once dashboards and alerts depend on a field, renaming it breaks them silently. Fields are a contract.
The trap
Structured logging plus "log everything" is more expensive than free text plus discipline. The structure makes each line larger and tempts teams to attach more context, while the improved queryability encourages retaining more for longer.
The benefit is real, and it is realised only when structure is paired with less volume, not more — using logs for the specific detail of specific events, and moving counting to metrics and causality to traces.