An observability vendor indexes logs by labels only, not full text, to control cost. Which queries become cheap, which become expensive, and how should teams structure logs and labels to fit?

The design choice

A conventional log system builds a full-text inverted index over log content, which makes arbitrary text search fast. It also means the index is often comparable in size to the data, and indexing dominates the cost of ingestion.

The label-only model — Loki's design — indexes a small set of labels and stores the log content as compressed chunks with no content index. Queries select a stream by labels, then brute-force scan the matching chunks for the content predicate.

The economic effect is large: ingestion becomes cheap because there is almost nothing to index, and storage is object storage rather than an indexed store.

Cheap versus expensive

Cheap: anything selecting by label within a bounded time range — one service, one namespace, one pod, one environment, over an hour. The label index narrows the scan to a small set of chunks, and the scan is fast and parallel.

Expensive: a content search across all streams over a long window, which becomes a scan of everything. The system parallelises aggressively and the cost is real, and it is proportional to bytes scanned rather than to results returned.

Catastrophic: high-cardinality labels. A label with many distinct values creates a stream per value, and the number of streams is the multiplicative product of all label cardinalities. Putting a request ID, user ID, trace ID or timestamp in a label explodes the index the design exists to keep small — and this is the single mistake that ruins the model.

How to structure logs for it

• Labels are for selection, not for information. Low cardinality, bounded, and known in advance: service, environment, namespace, level, region. A label is a filing-cabinet drawer, not a fact about the event.
• Everything high-cardinality goes in the log line, as structured JSON or key-value pairs. Request IDs, user IDs, durations, trace IDs. They remain searchable — by scanning within a narrow label selection, which is fast enough because the selection is narrow.
• Always query with a time bound and a label selector. A well-formed query for this system starts narrow and filters within; the anti-pattern is starting with a text predicate and hoping.
• Structured, parseable lines, so filters can be evaluated on parsed fields rather than by regular expression over raw text.
• Metrics derived from logs for anything queried repeatedly — a recurring log query is a metric that has not been created yet, and converting it removes the scan entirely.
• Consistent label taxonomy across teams, or queries cannot be written portably and dashboards fragment.

The judgement

This design trades query flexibility for ingestion cost, and it fits organisations whose logs are large, whose queries are usually scoped to a known service, and whose cost pressure is on ingestion.

It fits badly where investigation genuinely requires searching everything for an unknown string across a large corpus — security forensics being the clearest example. Choosing this model and then querying it like a full-text system produces both a bad experience and a large bill, and the fault is the mismatch rather than the system.