Intermittent Demand Forecasting

A spare part sells 0, 0, 0, 6, 0, 0, 3, 0, 0, 0, 0, 5 over twelve weeks. Simple exponential smoothing drops after every zero and jumps after every sale, so its forecast is always wrong in a characteristic way: too high right after a sale, too low just before the next. Worse, the forecast that minimises mean absolute error on this series is zero every week, which would stock nothing and lose every sale. Series like this are typical of product-level retail sales and spare parts, so in a large catalogue intermittent demand is an everyday case rather than an edge case.

Croston's decomposition

Croston's idea was to stop forecasting one noisy series and forecast two better-behaved ones: the size of demand when it occurs, \(z\), and the interval between demands, \(p\) (Croston, 1972, Forecasting and Stock Control for Intermittent Demands, JORS 23(3)). Both are smoothed only in periods with positive demand. If \(y_t > 0\) and \(q\) periods have elapsed since the last demand,

and otherwise both estimates are left unchanged. The forecast of demand per period is \(\hat{y} = \hat{z}/\hat{p}\).

Take \(\hat{z} = 4\), \(\hat{p} = 4\) and \(\alpha = 0.1\), then a demand of 6 arrives two periods after the last one. Size becomes 4.2, interval 3.8, and the forecast is \(4.2/3.8 \approx 1.105\) units per period. It then stays at exactly 1.105 through any run of zeros, however long.

Two corrections, two different problems

Bias. The forecast is a ratio of estimates, and \(\mathbb{E}[\hat{z}/\hat{p}] \ne \mathbb{E}[z]/\mathbb{E}[p]\). Because \(1/\hat{p}\) is convex, Jensen's inequality pushes the expected forecast above the true demand rate. Syntetos and Boylan showed the bias analytically (Syntetos & Boylan, 2001, On the bias of intermittent demand estimates, IJPE 71) and later proposed the Syntetos-Boylan Approximation (SBA), which multiplies Croston's forecast by \((1 - \alpha/2)\). In a comparison on 3,000 automotive-parts series against simple moving averages, simple exponential smoothing and Croston, the new estimator came out ahead (Syntetos & Boylan, 2005, The accuracy of intermittent demand estimates, IJF 21(2)). In the example, SBA forecasts \(0.95 \times 1.105 = 1.05\).

Obsolescence. Croston never updates during zeros, so a part that stops selling keeps its forecast indefinitely, and the inventory system keeps buying it. Teunter, Syntetos and Babai replaced the interval with a demand probability \(\pi\), updated every period (Teunter, Syntetos & Babai, 2011, Intermittent demand: Linking forecasting to inventory obsolescence, EJOR 214(3)):

with size still updated only on demand. Starting from \(\hat{\pi} = 0.25\) and \(\beta = 0.1\), the same demand of 6 lifts \(\hat{\pi}\) to 0.325 and the forecast to \(0.325 \times 4.2 = 1.37\). After ten consecutive zeros, \(\hat{\pi} = 0.325 \times 0.9^{10} \approx 0.113\) and the forecast has decayed to 0.48, while Croston's still reads 1.105. TSB is also unbiased for all points in time, which the authors note Croston's method is not.

The metric problem is sharper here

The general failure of percentage errors on zeros is covered under forecast evaluation at scale. Intermittent series add a subtler trap: absolute error is minimised by the median of the predictive distribution, and when more than half of periods are zero, the median is zero. A series with \(P(y = 0) = 0.7\) and mean demand of 0.9 units is best forecast by zero under MAE, although zero is useless for replenishment. The M5 organisers chose a squared-error scaled metric for exactly this reason, noting that absolute errors would reward forecasts close to zero when the goal was average sales (Makridakis, Spiliotis & Assimakopoulos, 2022, M5 accuracy competition: Results, findings, and conclusions, IJF 38(4)).

What M5 showed

M5 forecast 30,490 Walmart product-store series, aggregated into 42,840 across twelve hierarchy levels, and was the first M competition built around intermittency. The winning LightGBM ensemble improved on Croston's method by 45.6% averaged across levels, but the improvement was very uneven: 77.9% at the top level, 10.8%, 7.3% and 4.5% at levels 10, 11 and 12. At the product-store level, the most intermittent, the winner scored 0.884 against Croston's 0.926. Croston also beat seasonal naive at those three levels by 13.0%, 20.2% and 27.0%.

So the reading of M5 depends on which level you look at, and people disagree about it. One reading is that global gradient-boosted models with covariates have made classical intermittent methods obsolete. The organisers' own reading is narrower: machine learning won decisively where aggregation smooths the series and seasonality and covariates matter, while Croston and its variants still hold value at the product-store level.

When it breaks

A point forecast is the wrong output. Inventory decisions need a service level, which needs the distribution of lead-time demand. Croston-family methods give a mean rate, and bolting a distribution on afterwards, often Poisson or negative binomial, is a separate modelling step where much of the real error lives.

Lumpy demand breaks the size model. When sizes vary wildly as well as timing, both \(\hat{z}\) and its variance are poorly estimated from a handful of nonzero observations. A part with four sales a year gives four data points a year.

Smoothing parameters are hard to fit. With few demand events, optimised \(\alpha\) is unstable; low fixed values such as 0.1 are common in practice, which also means slow reaction to genuine change.

Temporal aggregation is an alternative with its own cost. Summing to weekly or monthly buckets removes zeros and lets ordinary methods work, but the forecast must then be disaggregated, and the right bucket size differs by series.