Business Metric Alerting

Technical monitoring measures whether requests are served: rate, errors, duration, saturation. Business metric alerting measures whether the product is working: listings created, checkouts completed, payments authorised, messages delivered.

The gap between them is a category of failure that technical monitoring cannot detect by construction.

The failure classes it catches

• Successful responses containing wrong content. Every technical signal is green.
• A workflow step silently skipped, where each individual request succeeds.
• A third-party integration accepting requests and doing nothing — returning 200 and producing no output.
• A feature flag or configuration change disabling a path, which is not an error, it is behaviour.
• A degraded upstream returning empty rather than failing. A search index returning zero results is a successful query, and a system that cannot distinguish "no results" from "no results because the index is empty" will miss a total outage of its most important surface.

Implementation patterns

• Alert on deviation from expected pattern, not on absolute thresholds. These metrics are strongly seasonal — hourly, daily, weekly, and around campaigns. A fixed threshold either fires every night or never fires. Compare against the same period in previous weeks with a tolerance band.
• Keep them in the operational monitoring system, on the same dashboards, with the same alerting. Split into an analytics tool with hourly refresh, they detect nothing — the entire value is operational latency.
• Segment by the dimensions that concentrate impact — region, platform, tenant size, customer tier.
• Pair each with a technical drill-down, so an alert on "checkouts down 30%" leads directly to the traces and service metrics for that path.
• Model the expected value, not just the recent average, so a planned campaign or a known holiday does not fire an alert and a genuinely anomalous quiet period does.

Industry example

A marketplace where sellers suddenly cannot list items, with every technical metric healthy, is the canonical case. The listing flow returned 200 at every step; a validation change silently rejected a field that most listings contained, and the rejection was rendered as a form message rather than an error response.

No technical signal moved. The only observable difference was that listings created per minute fell off a cliff — a metric that, alerted on with a seasonal baseline, would have detected the incident within minutes instead of hours.

The general property is that business metrics are closest to the thing that matters, so they catch every failure mode including ones nobody anticipated — a code bug, a misconfiguration, a third-party outage, or an unrelated change nobody connected to this surface.

They also make severity immediately legible to the business, which is what converts an engineering incident into an organisational response.

Failure scenarios

• Absolute thresholds on seasonal data, producing noise and then being muted.
• Business metrics in a separate analytics stack, arriving too late to be operational.
• No drill-down path, so the alert says something is wrong and offers no route to why.
• Metrics computed from a pipeline that shares a failure mode with the system being measured, so the detector fails with the thing it detects.
• Too many business metrics, so none is treated as authoritative.

Trade-offs

Business metrics are lagging and coarse: they tell you something is wrong, rarely what, and they need enough volume to be statistically meaningful — which makes them poor detectors for low-traffic surfaces and small tenants.

They are also owned ambiguously, sitting between engineering and product, and metrics with ambiguous ownership decay. Naming an owner is part of the implementation, not an afterthought.

They do not replace technical monitoring; they cover the blind spot it structurally has.

Interview question

"Your search backend starts returning empty results for 40% of queries. Every technical metric is healthy. Tell me which alert catches this, how you would set its threshold, and why an absolute threshold would not work."