A platform ingests high-volume telemetry from many sources. Which ingestion pattern properties matter most?

The properties

1. Decouple acceptance from processing. The ingestion path does one thing: durably accept and acknowledge. All expensive work happens asynchronously behind a log. Ingestion throughput is then bounded by write bandwidth rather than by the slowest consumer.

2. Backpressure rather than unbounded buffering. When consumers fall behind, signal upstream. An unbounded buffer converts a throughput problem into memory exhaustion and, worse, into worthless data — telemetry arriving hours late is not delayed, it is useless, and the system is spending money to deliver uselessness.

3. Idempotency via a producer-generated event id, since retries are inevitable and at-least-once is the only achievable delivery guarantee.

4. Schema enforcement at the boundary, with a registry and compatibility rules as a build gate — because agents run on customer infrastructure and upgrade slowly, so the platform must accept events from every version simultaneously, forever.

5. Partitioning by a key that gives useful ordering — per-source or per-customer — without a global ordering bottleneck. And per-partition monitoring, since aggregate throughput hides a saturated partition.

6. Aggregation at the edge, in the agent, before data crosses the network. The cheapest place to reduce volume is the earliest, and for high-cardinality telemetry this is the largest available lever.

The multi-tenant additions

Per-tenant quotas with a defined behaviour at the ceiling. When a tenant exceeds their budget, drop the offending dimension rather than the metric — preserving a usable aggregate is far better than losing the signal.

Detect unbounded dimensions automatically. A label whose distinct-value count grows roughly one per data point is an identifier, not a dimension, and it belongs in a different store.

Isolate the blast radius, so one tenant's volume degrades their own queries first rather than everyone's.

The principle

Any dimension a tenant controls will eventually be maximised, usually by accident. Every tenant-supplied input needs a bound, a meter and a graceful behaviour at the bound — and documentation is not a control, because it requires every current and future customer to read and follow it.