Autocorrelation and ARIMA
Reading the autocorrelation and partial autocorrelation functions to identify how much of a series is explained by its own past, and what the AR, I and MA components each represent.
The distinguishing feature of a time series is that observations are not independent, and the autocorrelation function measures exactly how. \(\rho_k = \mathrm{Corr}(y_t, y_{t-k})\) at each lag \(k\) is the raw material for classical model identification, and it also explains why every independence-assuming standard error computed on a time series is wrong.
ACF and PACF, and what each isolates
The ACF at lag \(k\) includes indirect effects. If \(y_t\) depends on \(y_{t-1}\) and \(y_{t-1}\) on \(y_{t-2}\), then \(y_t\) correlates with \(y_{t-2}\) through the chain even with no direct relationship.
The PACF at lag \(k\) is the correlation remaining after removing the effect of all shorter lags. It isolates the direct relationship, which is exactly what an autoregressive order needs.
The identification rules follow from this. An AR(\(p\)) process has a PACF that cuts off sharply after lag \(p\) and an ACF that decays gradually, because the direct dependence stops at \(p\) while indirect effects propagate. An MA(\(q\)) process is the mirror image: the ACF cuts off after \(q\) and the PACF decays, since a finite sum of past shocks has bounded correlation range. A mixed ARMA process has both decaying, which is precisely the case the visual rules cannot resolve and where information criteria take over.
The three components
with \(L\) the lag operator. Each letter is a different claim about the process.
AR(\(p\)): the value depends on its own recent values. This gives persistence and mean reversion; the process has memory of its level.
I(\(d\)): the series was differenced \(d\) times to reach stationarity. This is not a dynamic component but a statement that the model operates on changes.
MA(\(q\)): the value depends on recent shocks, the unpredictable innovations. This captures the situation where a surprise has an effect that persists for a few periods and then vanishes completely, which AR terms cannot express with a finite order.
SARIMA adds a seasonal \((P, D, Q)_m\) block multiplying the non-seasonal one, so a monthly series with annual seasonality gets terms at lags 12, 24 and so on.
Automatic order selection
auto.arima and its equivalents search \((p,d,q)(P,D,Q)_m\) using unit-root tests to choose \(d\) and \(D\), then minimising AICc over the remaining orders with a stepwise search. This is now the default in practice, and it is a genuine improvement over manual ACF reading for most series.
Two cautions. AIC comparisons are only valid across models with the same \(d\), since differencing changes the data being modelled and therefore the likelihood's scale; implementations handle this and hand-rolled comparisons often do not. And a stepwise search is not exhaustive, so it can miss the best model, which matters more for the small number of series where the fit is genuinely contested.
When it breaks
ARIMA is linear and homoscedastic. Threshold effects, regime switching and volatility clustering are outside what the model can represent, no matter the order. Fitting a higher order in response to a nonlinear pattern produces a complicated model that still cannot express the behaviour.
One series, one model. Fitting ARIMA independently to 50,000 SKUs is 50,000 estimations that share nothing, and each is fitted on a short, noisy history. Global models, which train one model across all series, are the modern alternative and the reason cross-learning came to dominate forecasting competitions.
Exogenous regressors need care. ARIMAX includes covariates, and their coefficients are interpretable only if the differencing is applied consistently to them and the errors' dynamics are correctly specified. A regression with ARIMA errors and an ARIMAX model are different specifications with different interpretations, and they are routinely conflated.
Long-horizon forecasts converge to a constant. A stationary ARIMA's forecast reverts to the mean as horizon grows, with the interval widening to the unconditional variance. That is honest behaviour, and it means the model contributes nothing beyond a few multiples of its memory length. Any longer horizon needs an external driver, not a higher order.
7 flashcards for this concept
Click a card to reveal the answer.