intermediate 2 min answer

A food-delivery marketplace in the mould of Swiggy publishes its 07:00 revenue dashboard from a six-stage chain: ingest, bronze, silver, gold, metrics, extract. Each stage independently finishes on time 99% of the days, and a miss is not recovered the same night. What is the chance the dashboard is on time, and what does the number say about how to design the SLA?

slafreshnessreliabilityserial dependencyerror budgetestimation
Show the full answer Hide the answer

The assumptions, stated

Stages are in series (each needs the previous one), failures are independent, and a stage that misses causes the dashboard to miss. The per-stage figure of 99% is the stage's own on-time rate and already includes its own retries.

The arithmetic

P(on time) = 0.99^6 = 0.941. That is about 21 late mornings a year (365 x 0.059), from a chain in which every stage looks excellent in isolation. At 98% per stage the dashboard lands 88.6% of days, roughly 42 late mornings.

A reliability budget of 99% for the final product, spread across six serial stages, needs each stage near 99.83% (0.99^(⅙)). That is about 16 hours of lateness a year per stage, a target most batch stages do not hit without engineering for it.

The number, with its range

Failures are rarely independent: a shared cluster outage or a source delay hits several stages at once, which makes the real distribution better than the independent estimate on average and worse in its worst nights, because misses cluster. Treat 94% as a ceiling for the chain, not the expected value.

Which assumption dominates the error

Independence. Count how many stages share the same cluster, orchestrator, warehouse, or upstream source. Shared fate means the true on-time rate is closer to the worst shared component than to the product.

What the number rules in or out

  • It rules out committing to a 99% product SLA on a six-stage serial chain built from 99% stages; the arithmetic cannot deliver it.
  • It rules in three levers, in order of cost: fewer stages (collapse silver and gold if no consumer reads silver), slack (set stage deadlines with headroom so a 20-minute slip is absorbed, which converts late nights into on-time ones), and partial publish (publish yesterday's numbers with a freshness stamp when a stage misses, instead of nothing).
  • It rules in a freshness SLO on the product, such as 95% of days by 07:00 and every day by 09:00, rather than per-stage targets that nobody can aggregate.

What it costs

Slack means starting stages earlier or sizing clusters larger to finish sooner, and fewer stages gives up clean layering. Partial publish needs consumers to read the staleness stamp; a dashboard that shows stale numbers without saying so is worse than a late one.

When this analysis is overkill

A weekly report read by three people tolerates a missed morning. Do this arithmetic when a late artefact has a cost: a finance close, a model retrain, a partner feed with a contractual deadline.

Common weak answers

  • "99% each is fine, so 99% overall." Probabilities of serial stages multiply.
  • "Add alerts to every stage." Alerts shorten recovery; they do not change how often a stage is late.