Stationarity and Differencing

Fit a regression of one random walk on another and you will find a highly significant relationship with a large \(R^2\) roughly three quarters of the time, even though the two series are independent by construction. This is spurious regression, and it is not a small-sample artefact: the t-statistic diverges as the sample grows. Non-stationarity is the reason, and it is why the first question about any time series is whether its statistical properties are stable.

A series is weakly stationary if its mean, variance and autocovariance structure do not depend on time. Only the autocovariance's dependence on the lag is allowed, not on where in the series you look. Almost every classical method assumes this, and most real series violate it.

Trends, unit roots, and why the distinction matters

Two different mechanisms produce an upward-drifting series, and they need opposite treatments.

Trend stationary: \(y_t = \alpha + \beta t + \varepsilon_t\). The series fluctuates around a deterministic line, and shocks are temporary. Subtracting the fitted trend leaves stationary residuals.

Difference stationary (unit root): \(y_t = y_{t-1} + \varepsilon_t\). The series has no fixed level to return to, and shocks are permanent, accumulating forever. Differencing leaves stationary residuals; detrending does not.

Applying the wrong one is a real error. Detrending a unit-root process leaves residuals that are still non-stationary; differencing a trend-stationary process introduces a moving-average unit root and inflates variance. The ADF and KPSS tests approach this with opposite nulls, ADF testing a unit root as the null and KPSS testing stationarity as the null, so running both and reading the pair is standard practice. Their agreement is informative and their disagreement, which is common, means the evidence is weak.

Seasonal non-stationarity needs seasonal differencing, \(y_t - y_{t-m}\) for period \(m\). First differencing a monthly series does not remove an annual cycle.

Differencing changes the question

\(d = 1\) means the model forecasts changes, not levels, and the forecast must be integrated back to produce a level. Two consequences follow that are easy to miss.

Forecast uncertainty grows without bound. For a random walk, the \(h\)-step variance is \(h\sigma^2\), so the interval widens as \(\sqrt{h}\) forever. That is the correct answer for a unit-root process and it looks alarming next to a trend-stationary model's bounded intervals, which are only correct if the trend really is deterministic.

Over-differencing has a cost. Each difference removes a degree of freedom, amplifies high-frequency noise, and introduces negative autocorrelation at lag 1 that the model must then absorb. \(d = 2\) is occasionally right and usually a sign that a transformation was needed instead: a variance-stabilising log or Box-Cox often removes the appearance of a changing level entirely.

When it breaks

Cointegration means differencing throws away the relationship. Two non-stationary series can share a common stochastic trend, so a linear combination of them is stationary. Differencing both destroys the long-run equilibrium relationship that was the interesting part. Error-correction models keep it, and this is the whole point of cointegration analysis in econometrics.

Structural breaks masquerade as unit roots. A series that is stationary around one level, jumps once, and is stationary around a new level will fail a unit-root test, because the test cannot distinguish a permanent shock from a single regime change. Differencing then treats a one-off event as ongoing dynamics. Tests allowing for a break exist; looking at the plot catches most cases faster.

Stationarity is not required by every method. Tree ensembles on lagged features, and modern neural forecasters, do not assume it formally. They still cannot extrapolate a trend beyond the training range, which is a different failure with the same practical consequence, so differencing remains standard even where the assumption is not.

Variance non-stationarity is separate. A series whose level is stable but whose volatility clusters is still non-stationary, and differencing does nothing for it. That is the domain of ARCH and GARCH models, and it is the usual situation for financial returns.