Device Telemetry at Scale
Deciding what a device sends, how often, and what is aggregated before it leaves.
4 to work through
-
advanced
A fleet platform emits 25 metric series per device, tagged with device id, for 400,000 devices. The observability bill is now larger than the compute bill for the platform itself. The team proposes dropping the device id tag and keeping per-model and per-region aggregates. What does that buy, what does it give up, and when does the bill arrive?
3 min answer -
advanced
A logistics platform receives location updates from a large fleet every few seconds. Where should volume be reduced, and what is the ordering of leverage?
2 min answer -
advanced
A ride-hailing platform receives location updates from hundreds of thousands of active drivers every few seconds. Design the ingest and serving path.
2 min answer -
advanced
Design telemetry collection where the volume of monitoring data would cost more than the systems being monitored.
2 min answer
5 terms in this topic
Conflatable Classification
Explicitly classifying each telemetry stream as superseding or discrete, because conflation is the correct overload behaviour for one and silent data…
conceptDevice Telemetry at Scale
The volume problem created by many devices each emitting small frequent messages, where the answer is filtering at the edge rather than ingesting eve…
patternEdge Aggregation
Summarising or filtering telemetry at the device or gateway so that only what is needed leaves, which is usually the difference between a viable and …
conceptFleet Reconnection Storm
A synchronised burst of devices re-establishing connections after any disruption, which is far more expensive than steady-state traffic and routinely…
metricPer-Device Series Budget
The number of distinct metric time series a fleet creates - series per device multiplied by device count - which decides observability cost long befo…
Neighbouring topics
Edge, Mobile & IoT
General material on architecture beyond the data centre boundary.
Mobile App Architecture
Layering, navigation, background execution, and the platform rules you do not set.
Offline-First
Treating connectivity as an optimisation, with a local store as the source of truth.
Sync & Conflict Resolution
Two devices that both changed the same record while neither could see the other.
CRDTs
Data types that converge without coordination, and the semantics you must accept.
Mobile Release Strategy
Store review, staged rollout, and supporting versions you can never force off.
Push & Background Work
Delivery that is best-effort, and an operating system that will kill your process.
Device Identity
Identifying a thing rather than a person, and rotating a credential you cannot type.
Edge Compute Topologies
Regional, metro, on-premises and on-device, and what each tier is genuinely for.
Edge Functions
Short-lived compute at the CDN, its runtime limits, and what must stay at origin.
Edge Data Consistency
Replicated read state at hundreds of locations, and writes that still go to one.
IoT Ingest Architecture
Millions of small, unreliable, frequently duplicated messages arriving continuously.
Device Provisioning
Getting identity and configuration onto hardware at manufacture or first boot.
Fleet Management
Inventory, health, configuration and grouping across devices you will never see.
OTA Updates
Updating firmware over a flaky link, with rollback, and without bricking the device.
Constrained Protocols
MQTT, CoAP and their kin, chosen for power, packet size and intermittent links.
Digital Twin
A server-side model of a physical thing's reported and desired state.
Edge Security & Attestation
Secure boot, hardware roots of trust, and proving what is running on a device.
Physical-World Failure Modes
Power loss, tampering, clock drift, thermal limits, and a truck through the fibre.