The Kalman Filter: Sixty-Six Years of Bayes' Rule, One Observation at a Time

The first test run took a little over an hour on an IBM 704, and the answer was wrong. It was early 1961 at NASA Ames, and Stanley Schmidt's eight-person analysis branch had spent months turning Rudolf Kalman's new paper into a simulation of navigation around the Moon. Then someone found a wrongly called subroutine, and "on the second run everything worked fine" (McGee & Schmidt, 1985, NASA TM-86847). What they validated is, in different clothes, the recursion R's arima() uses today: "the innovations and their variance found by a Kalman filter" (R stats::arima documentation).

The Kalman filter is Bayes' rule applied one observation at a time in the one model family where Bayes' rule stays closed-form forever: linear dynamics, Gaussian noise. Exact likelihoods, missing-data handling, smoothing and forecast intervals fall out of that closure; outliers, nonlinearity, misspecified noise and round-off are ways it fails. Every successor, from particle filters to Mamba, chooses which part of the closure to give up.

Why this matters: ARIMA estimation, exponential smoothing with prediction intervals, structural models, nowcasting and causal impact analysis all run on the same predict-update recursion. The derivation is what lets you diagnose a diverging filter, a variance estimated at zero, or a claim that a neural sequence model "is a Kalman filter".

TL;DR

• The update is Gaussian conditioning: the gain \(K = PZ^\top(ZPZ^\top + H)^{-1}\) is the regression coefficient of hidden state on observation.
• In the local level model the steady-state gain depends only on \(q = Q/H\): 0.095 at \(q = 0.01\), 0.618 (one over the golden ratio) at \(q = 1\), 0.916 at \(q = 10\). Exponential smoothing's \(\alpha\) is this gain.
• A missing observation is exact and free: zero gain, variance grows by \(Q\), no likelihood term.
• The log-likelihood is a sum over \(n\) one-step prediction errors, so maximum likelihood for ARIMA or ETS is an optimiser wrapped around the filter.
• Apollo carried a square-root Kalman filter, but the Ames studies backed ground radar as the primary navigation source with the onboard system as backup.
• A particle filter on a 200-dimensional Gaussian toy problem needed at least \(10^{11}\) particles in one analysis (Snyder et al., 2008).
• The steady-state filter mean is a linear time-invariant recurrence: S4 and Mamba kept that recurrence and dropped the covariance.

At a Glance

flowchart LR
    PRIOR["Belief: mean a, variance P"] --> PRED["Predict: apply T, add Q"]
    PRED --> INNOV["Innovation v = y - Za"]
    OBS["Observation y_t"] --> INNOV
    INNOV --> GAIN["Gain K from P and H"]
    GAIN --> UPD["Update: a + Kv, shrink P"]
    UPD -->|"next step"| PRIOR
    INNOV --> LIK["Log-likelihood term"]
    UPD --> SMOOTH["RTS smoother, backward pass"]
    UPD --> FC["Forecasts with intervals"]

    classDef blue fill:#1e40af,stroke:#3b82f6,stroke-width:1px,color:#fff
    classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
    classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff
    classDef amber fill:#b45309,stroke:#fbbf24,stroke-width:1px,color:#fff

    class OBS,PRIOR blue
    class PRED,INNOV,GAIN,UPD purple
    class LIK amber
    class SMOOTH,FC teal

Prediction adds uncertainty; the update removes it. The innovation, the surprise between prediction and arrival, feeds the update, the likelihood and the smoother.

Before Kalman, and the Apollo Story Told Carefully

The recursion is older than Kalman. In 1880 the Danish astronomer Thorvald Thiele modelled a series as regression plus Brownian motion plus white noise, derived a recursive estimator, and estimated the two variances with an iteration that is essentially EM; Lauritzen's reading is that Thiele computed what we now call Kalman filtering and smoothing (Lauritzen, 1981, International Statistical Review 49(3)). Kalman's 1960 contribution was the general discrete-time solution in state-space form, dropping the Wiener filter's stationarity requirement (Kalman, 1960, Journal of Basic Engineering 82(1)); the continuous-time version with Bucy followed in 1961 (Kalman & Bucy, 1961).

timeline
    title Recursive estimation from Thiele to selective state spaces
    1880 : Thiele derives a recursive filter for Brownian motion plus noise
    1960 : Kalman publishes the discrete-time filter
         : Kalman presents it at NASA Ames in the fall
    1961 : Ames validates the extended Kalman filter for lunar navigation
         : Battin at MIT takes it into Apollo studies
    1965 : Rauch, Tung and Striebel publish the smoother
         : Schweppe writes the likelihood through innovations
    1979-80 : Exact ARMA likelihood by Kalman filtering
            : Jones fits ARMA models with missing data
    1993-94 : Gordon, Salmond and Smith publish the particle filter
            : Evensen proposes the ensemble Kalman filter
    2002 : Exponential smoothing recast as state-space models
    2021 : S4 and parallel-scan Bayesian smoothers
    2023 : Mamba makes SSM parameters input dependent

The popular line, "the Kalman filter flew Apollo to the Moon", compresses three facts, all from McGee and Schmidt's first-hand account. Kalman visited Schmidt at Ames in the fall of 1960, unaware of the lunar work. Schmidt saw that the linear theory could be applied to the nonlinear problem by linearising, and the team then relinearised about the current estimate instead of a nominal trajectory: the extended Kalman filter (EKF). They also split the algorithm into time and measurement updates so sightings could arrive at irregular times, "a major and critical step forward" in their words.

Schmidt told Richard Battin at MIT's Instrumentation Laboratory in early 1961, and James Potter, working with Battin, devised the first square-root formulation, used for Apollo because it survived the flight computer's short word length (Potter & Stern, 1963, AIAA Guidance and Control Conference). But the Ames follow-up studies of 1962 to 1964 found ground radar generally superior to onboard optics, which "supported the ultimate decisions to have the primary Apollo navigation conducted from the ground ... with a backup system on the spacecraft." By McGee and Schmidt's account, the first real-time airborne Kalman filter was the C-5A transport's navigation system later that decade (Grewal & Andrews, 2010, IEEE Control Systems Magazine 30(3) survey the aerospace applications that followed).

[IMAGE: Three-lane history strip. Lane "Theory": Thiele 1880, Kalman 1960, Kalman-Bucy 1961. Lane "Ames and MIT": fall 1960 visit, early 1961 IBM 704 simulation, Battin at MIT, Potter square-root filter. Lane "Deployment": Apollo onboard backup with ground radar primary, C-5A first real-time airborne filter, RAINPAL 1972 flight test on a 24-bit XDS-920. Caption: "Linearisation, the time and measurement split, and square-root arithmetic were all added after the 1960 paper."]

Statisticians arrived by another road. Schweppe showed the Gaussian likelihood could be evaluated through innovations (Schweppe, 1965, IEEE Trans. Information Theory 11(1)); Harvey and Phillips used state-space form for exact ARMA-disturbance likelihoods (Harvey & Phillips, 1979, Biometrika 66(1)); Jones handled missing observations the same way (Jones, 1980, Technometrics 22(3)); and Harvey's 1989 book made structural level-trend-season models a school of their own (Harvey, 1989).

How the Kalman Filter Actually Works

The model and the one lemma

In Durbin and Koopman's notation (Durbin & Koopman, 2012):

The hidden state \(\alpha_t\) has dimension \(m\). \(Z\) says what you see of it, \(T\) how it evolves, \(H\) is measurement noise and \(Q\) is how much the state wanders. Write \(a_t, P_t\) for the mean and covariance of \(\alpha_t\) given data to \(t-1\), and \(a_{t|t}, P_{t|t}\) after seeing \(y_t\). The derivation needs one fact: if \(x\) and \(y\) are jointly Gaussian, then

The posterior variance does not depend on the observed value, only on the fact of observation. That is why the Ames team could run error analyses for missions that had never flown.

Update and prediction

Given the past, \(y_t\) has mean \(Za_t\), covariance \(F_t = ZP_tZ^\top + H\) and cross-covariance \(P_tZ^\top\) with the state. Substituting into the lemma:

With \(Z = 1\) the gain is \(P_t/(P_t + H)\), the share of predicted uncertainty that belongs to the state. The information form makes the averaging explicit:

Precisions add and means are precision-weighted. The same \(a_{t|t}\) minimises a quadratic penalty on distance from prior and from data, so without Gaussianity the filter is still the best linear estimator, and recursive least squares is a Kalman filter with a constant state (Sayed & Kailath, 1994, IEEE Signal Processing Magazine 11(3)). Prediction pushes the posterior through the dynamics:

Prediction creates uncertainty; update destroys it. At balance, the local level model (\(Z = T = R = 1\)) has predicted variance solving \(\bar{P}^2 - Q\bar{P} - QH = 0\). With \(q = Q/H\),

so signal-to-noise ratio alone fixes the long-run gain. At \(q = 1\), \(\bar{p} = \varphi\) and \(K_\infty = 1/\varphi \approx 0.618\).

ARIMA and ETS are choices of matrices

In steady state \(a_{t+1} = a_t + K_\infty v_t\), which is simple exponential smoothing with \(\alpha = K_\infty\). The ETS framework uses a single source of error instead, \(y_t = \ell_{t-1} + e_t\) and \(\ell_t = \ell_{t-1} + \alpha e_t\), and gets likelihoods, intervals and automatic selection across the exponential smoothing family (Hyndman et al., 2002, International Journal of Forecasting 18(3)). Both forms reduce to ARIMA(0,1,1), but Harvey's two-noise version confines \(\alpha\) to \([0, 1]\) while the single-source form admits a wider region. The better default remains a difference of school.

For ARMA(\(p\), \(q\)), let \(r = \max(p, q+1)\) and use an \(r\)-dimensional state with

padding unused \(\phi_i\) and \(\theta_j\) with zeros. All randomness enters through the state, so \(H = 0\). Integrated components get a diffuse prior, which is what R's kappa argument controls.

The likelihood, missing data and smoothing

By the chain rule the joint density is a product of one-step predictions, each \(\mathcal{N}(Za_t, F_t)\):

Estimation proposes parameters \(\psi\), runs the filter, reads \(\log L\) and lets a quasi-Newton optimiser propose better \(\psi\); exact ARMA likelihood was published as Applied Statistics algorithm AS 154 in exactly this way (Gardner, Harvey & Phillips, 1980).

If \(y_t\) is missing, set \(K_t = 0\), carry \(a_t\) and \(P_t\) forward, and omit the term. For partly missing vectors, delete the matching rows of \(Z\), \(y_t\) and \(H\). R's documentation says missing values "are handled exactly in method 'ML'."

Smoothing asks what to believe about \(t\) once the whole sample is in. Condition \(\alpha_t\) on \(\alpha_{t+1}\) using their joint Gaussian given data to \(t\), then average over the smoothed \(\alpha_{t+1}\):

This backward pass is the Rauch-Tung-Striebel smoother (Rauch, Tung & Striebel, 1965, AIAA Journal 3(8)). It also supplies the expectations EM needs, so Thiele's variance iteration lives on.

Beyond linear and Gaussian

The EKF swaps \(T\) and \(Z\) for Jacobians at the estimate. The unscented filter pushes \(2m+1\) sigma points through the nonlinear maps and refits moments; Julier and Uhlmann called the EKF "only reliable for systems that are almost linear on the time scale of the updates" (Julier & Uhlmann, 2004, Proceedings of the IEEE 92(3)). Gustafsson and Hendeby then showed a less-cited second-order EKF is closely related to the UKF (Gustafsson & Hendeby, 2012, IEEE Trans. Signal Processing 60(2)). The particle filter drops Gaussianity, reweighting samples by \(p(y_t \mid x_t)\) and resampling (Gordon, Salmond & Smith, 1993, IEE Proceedings F 140(2)). The ensemble Kalman filter keeps the Gaussian update but estimates covariance from model runs (Evensen, 1994, JGR 99(C5)).

[IMAGE: Four panels over the same banana-shaped posterior. EKF: tangent-line ellipse offset from the banana. UKF: five sigma points and a refitted ellipse nearer the true mean. Particle filter: 500 weighted dots tracing the banana. EnKF: 40 members and their sample-covariance ellipse. Caption: "Four answers to one question: how much of the Gaussian closure will you give up?"]

Seeing It in Motion

In practice filtering, likelihood, optimisation and smoothing interleave; a missing observation changes one message.

sequenceDiagram
    participant O as Optimiser
    participant D as Data stream
    participant F as Kalman filter
    participant L as Log-likelihood
    participant S as RTS smoother
    O->>F: Trial parameters H and Q
    D->>F: y1 arrives
    F->>L: Add term from v1 and F1
    D->>F: y2 is missing
    Note over F: Gain zero and P grows by Q
    D->>F: y3 arrives
    F->>L: Add term from v3 and F3
    L->>O: Total log-likelihood
    O->>F: Improved parameters, rerun
    F->>S: Stored filtered and predicted moments
    S->>S: Backward pass from n to 1

The right approximation depends on which assumption fails and on state size.

flowchart TB
    START{"Linear and Gaussian?"}
    START -->|"yes"| KF["Kalman filter, exact posterior"]
    KF --> EXTRA["Exact likelihood and smoother"]
    START -->|"no"| MILD{"Smooth and unimodal?"}
    MILD -->|"yes"| SIGMA["EKF or UKF"]
    MILD -->|"no"| DIM{"Small state dimension?"}
    DIM -->|"yes"| PF["Particle filter"]
    DIM -->|"no"| ENKF["Ensemble Kalman filter"]
    START -->|"dynamics unknown"| LEARN["Learned SSM, S4 or Mamba"]
    PF --> RISK["Weight collapse in high dimension"]
    LEARN --> LOSS["No calibrated posterior"]

    classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
    classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff
    classDef rose fill:#be123c,stroke:#fb7185,stroke-width:1px,color:#fff
    classDef slate fill:#334155,stroke:#64748b,stroke-width:1px,color:#e2e8f0

    class START,MILD,DIM slate
    class KF,SIGMA,PF,ENKF,LEARN purple
    class EXTRA teal
    class RISK,LOSS rose

[IMAGE: Two stacked panels of a 60-point local level series with a gap from t = 30 to 38. Top: filtered mean whose 95% band widens steadily through the gap and snaps back when data return. Bottom: smoothed mean whose band widens most at the gap's centre. Caption: "Filtering widens monotonically across missing data; smoothing uses both edges, so uncertainty peaks mid-gap."]

By the Numbers

Sources: Lauritzen (1981); McGee & Schmidt (1985); Julier & Uhlmann (2004); Snyder et al. (2008); Hartikainen & Särkkä (2010); Särkkä & García-Fernández (2021); Gu & Dao (2023); Makridakis, Spiliotis & Assimakopoulos (2018). The Mamba figure is the authors' own claim. The second table is computed from the Riccati equation above; "memory" is a heuristic.

Moving \(q\) from 0.01 to 0.1 nearly triples the gain; moving from 1 to 10 adds only 0.3. A fitted smoothing weight near 0.1 says the level barely moves; near 0.9, the series is nearly a random walk.

[IMAGE: Line chart of steady-state gain (0 to 1) against log10 q from -3 to 2, the five table points marked and the q = 1 point annotated 0.618. Caption: "Exponential smoothing's alpha is a sigmoid in log signal-to-noise ratio, a property of the series rather than a free dial."]

A Concrete Example

A daily sensor follows a local level model with \(H = 2\), \(Q = 0.5\) (so \(q = 0.25\)) and prior \(a_1 = 100\), \(P_1 = 4\). Readings: \(y_1 = 103\), \(y_2 = 101\), \(y_3 = 104\).

Step 1, day 1. \(F_1 = 4 + 2 = 6\), \(v_1 = 3\), \(K_1 = 4/6 = 0.667\). Filtered level \(100 + 0.667 \times 3 = 102.00\), variance \(4 \times (1 - 0.667) = 1.333\). A vague prior lets the reading pull two-thirds of the way.

Step 2, predict day 2. \(a_2 = 102.00\), \(P_2 = 1.333 + 0.5 = 1.833\).

Step 3, day 2. \(F_2 = 3.833\), \(v_2 = -1\), \(K_2 = 0.478\). Level \(101.52\), variance \(0.957\). The gain fell 28% because the filter now trusts itself more.

Step 4, day 3. \(P_3 = 1.457\), \(F_3 = 3.457\), \(v_3 = 2.48\), \(K_3 = 0.421\). Level \(101.52 + 0.421 \times 2.48 = 102.57\), variance \(0.843\). Gains \(0.667, 0.478, 0.421\) are converging on the table's \(0.390\).

Step 5, likelihood. Each day adds \(-\tfrac{1}{2}(\log 2\pi + \log F_t + v_t^2/F_t)\):

An optimiser would try other \((H, Q)\) pairs to raise \(-6.714\); over a long series, standardised innovations should look like independent standard normals.

Step 6, day 4 is missing. \(a_4 = 102.57\), \(P_4 = 1.343\); gain zero, no likelihood term. Day 5 prediction: \(a_5 = 102.57\), \(P_5 = 1.843\), and a 95% interval for the day 5 reading of \(102.57 \pm 1.96\sqrt{1.843 + 2} = 102.57 \pm 3.84\).

Step 7, smooth backwards. Day 2: \(J_2 = 0.957/1.457 = 0.657\), so \(a_{2|3} = 101.52 + 0.657 \times 1.05 = 102.21\) and \(P_{2|3} = 0.957 + 0.657^2 \times (0.843 - 1.457) = 0.692\). Day 1: \(J_1 = 1.333/1.833 = 0.727\), \(a_{1|3} = 102.15\), \(P_{1|3} = 0.730\).

The high day 3 reading revised day 2 upward by 0.69 and cut its variance 28%. That is exactly why smoothed states in a backtest leak the future.

[IMAGE: Days 0 to 5. Grey dots at 103, 101, 104 and a hollow marker for missing day 4. Blue steps for filtered levels 102.00, 101.52, 102.57 with two-sigma bands; teal line for smoothed levels 102.15, 102.21, 102.57 with narrower bands; an arrow on day 2 reading "revised up 0.69 after day 3". Caption: "The filter reacts, the smoother reconsiders, and the gap simply widens the band."]

Where It Breaks

Divergence from an overconfident covariance

At Ames, after a run of accurate measurements, \(P\) became so small that new measurements "would be essentially ignored", and the estimate drifted from the truth while reporting tiny uncertainty (McGee & Schmidt, 1985). The mechanism is the gain: \(P/(P+H)\) goes to zero with \(P\), and \(P\) shrinks too far when \(Q\) understates real model error. Their fix, "pseudonoise" in the time update, is still the fix. The symptom is in the innovations: standardised innovation variance well above one means the filter is diverging, however plausible the states look.

Finite precision breaks the covariance

\(P_{t|t} = (I - K_tZ)P_t\) subtracts nearly equal matrices, and round-off can leave \(P\) asymmetric or indefinite. Ames happened to multiply in the stable order and only met the problem when the order was changed. The Joseph form, \((I - K_tZ)P_t(I - K_tZ)^\top + K_tHK_t^\top\), is a sum of positive semidefinite terms and stays valid even for suboptimal gains. Square-root filters from Potter onward propagate \(S\) with \(P = SS^\top\), roughly halving the dynamic range the arithmetic must hold. It still matters for large or badly scaled states.

One outlier drags the state

Under Gaussian noise a ten-sigma innovation is essentially impossible, so the filter believes it. With \(K_\infty = 0.39\), a spike of \(+20\) moves the level by \(7.8\), and the error decays only as \(0.61^k\): still \(1.8\) three steps later. Schmidt's Lockheed team gated innovations at three sigma in 1961, which also rejects genuine level shifts. The principled alternative uses heavy-tailed observation densities and bounded influence, a line starting with Masreliez & Martin (1977), IEEE Trans. Automatic Control 22(3), and gives up exactness to get it.

Likelihoods that pin a variance at zero

The likelihood is often flat along the \(Q/H\) direction in short series, and it can peak at the boundary \(Q = 0\): the level freezes, the steady gain goes to zero, and forecasts ignore recent data. In ARIMA(0,1,1) terms this is a moving-average root on the unit circle, a feature of the finite-sample likelihood, not an optimiser bug. Remedies are priors on the variances (the Bayesian structural route behind CausalImpact, Brodersen et al., 2015, Annals of Applied Statistics 9(1)), fixing the ratio from domain knowledge, or checking whether the zero survives across backtest windows.

Nonlinearity, dimension and cost

EKF Jacobians are evaluated at the estimate, so a bad estimate yields a bad linearisation. A single Gaussian cannot represent a bimodal posterior at all. Particle filters handle multimodality only in low dimension, given the \(10^{11}\)-particle result above. Dense states cost \(O(m^3)\) per step, and the time recursion is serial, a limit Särkkä and García-Fernández removed for linear Gaussian models with a parallel scan of logarithmic span (Särkkä & García-Fernández, 2021, IEEE Trans. Automatic Control 66(1)).

[IMAGE: Two rows. Top: truth and estimate separating after step 40 while the reported band shrinks to a line, with standardised-innovation variance climbing from 1 to 6 underneath. Bottom: same data with Q inflated fivefold, estimate tracking and innovations near unit variance. Caption: "Divergence looks like confidence. The innovations, not the states, reveal it."]

Alternative Designs

Neural filters such as KalmanNet (Revach et al., 2022, IEEE Trans. Signal Processing 70) report gains under model mismatch, the regime a practitioner would first attack by estimating \(H\) and \(Q\).

The link to modern sequence models is exact at one level. Substitute the steady-state update into the prediction:

This is a linear time-invariant system with \(\bar{A} = T(I - K_\infty Z)\) and \(\bar{B} = TK_\infty\), the same shape as an S4 layer's \(h_t = \bar{A}h_{t-1} + \bar{B}x_t\), whose matrices come from a discretised continuous-time system learned by gradient descent (Gu, Goel & Ré, 2022, ICLR). Mamba makes the step size and \(B\), \(C\) input dependent (Gu & Dao, 2023), which resembles a time-varying gain, and Longhorn derives its recurrence from an online-learning objective (Liu et al., 2024, arXiv:2407.14207), the same move that yields recursive least squares. The kinship is structural, not identity: none of these layers carries a covariance, a noise model or a posterior, and "Mamba is a Kalman filter" discards precisely what makes the filter useful for forecasting.

How It Is Used in Practice

Forecasting software. R's arima() computes exact likelihood by Kalman filtering with Gardner-Harvey-Phillips initialisation and diffuse priors; ETS implementations rest on the single-source state-space framework; KFAS brings Durbin-Koopman filtering to exponential-family models (Helske, 2017, Journal of Statistical Software 78(10)).

Nowcasting and causal impact. Dynamic factor models combine macroeconomic releases published on different days, a "ragged edge" that is just a missing-data pattern to the filter (Giannone, Reichlin & Small, 2008, Journal of Monetary Economics 55(4)). CausalImpact fits a Bayesian structural time series before an intervention and uses its posterior predictive as the counterfactual.

Competitions. In the M4 competition, twelve of the seventeen most accurate methods were combinations of mostly statistical approaches, and the winner hybridised exponential smoothing with a recurrent network (Makridakis et al., 2018, IJF 34(4); Smyl, 2020, IJF 36(1)).

Insights Worth Remembering

1. The Kalman gain is a regression coefficient. Every intuition about the gain is an intuition about regressing the hidden state on the observation, given the past.
2. Posterior variance ignores the data values. It depends only on which observations exist. That enabled pre-flight error analysis and also lets a misspecified filter be confidently wrong.
3. Exponential smoothing is a Kalman filter that forgot its derivation. Its \(\alpha\) is the steady-state gain, set by signal-to-noise ratio, and equals \(1/\varphi\) when \(Q = H\).
4. Missing data is native, not an edge case. Skipping the update is exact; imputing then filtering asserts information the system never received.
5. Apollo navigated primarily from the ground. The Ames studies that proved the filter also supported radar as primary, which separates history from legend.
6. Most filter failures are covariance failures. Divergence, round-off, outliers and bad initialisation show up in \(P\) and \(F\) before the state. Watch the innovations.
7. Modern SSMs kept the recurrence and dropped the probability. What they dropped, calibrated uncertainty, is what forecasters need most.

Open Questions

Can learned sequence models recover calibrated uncertainty at scale? Hybrids such as KalMamba exist (Becker, Freymuth & Neumann, 2024, arXiv:2406.15131), but calibrated intervals on forecasting benchmarks are not established.

How much reported UKF and neural-filter advantage survives a well-tuned baseline? Gustafsson and Hendeby suggest the UKF-EKF gap is smaller than advertised. A systematic real-data comparison against classical filters with maximum likelihood noise estimates appears to be missing.

Single source or multiple sources of error? Both yield ARIMA reduced forms with different admissible regions. Whether the wider single-source region helps or overfits short business series is unmeasured.

What does the test-time regression view give time series? Wang, Shi and Fox recast many sequence layers as online regressions over associative memory (Wang, Shi & Fox, 2025, arXiv:2501.12352), placing them beside recursive least squares. Whether adding state noise and covariance tracking would improve their forecasts is speculation.

Sources and Further Reading

1. Kalman, R. E. (1960). "A New Approach to Linear Filtering and Prediction Problems." J. Basic Engineering 82(1), 35-45. doi:10.1115/1.3662552
2. McGee, L. A., & Schmidt, S. F. (1985). "Discovery of the Kalman Filter as a Practical Tool for Aerospace and Industry." NASA TM-86847. NTRS
3. Grewal, M. S., & Andrews, A. P. (2010). "Applications of Kalman Filtering in Aerospace 1960 to the Present." IEEE Control Systems Magazine 30(3), 69-78. doi:10.1109/MCS.2010.936465
4. Lauritzen, S. L. (1981). "Time Series Analysis in 1880: A Discussion of Contributions Made by T. N. Thiele." International Statistical Review 49(3), 319-331. doi:10.2307/1402616
5. Rauch, H. E., Tung, F., & Striebel, C. T. (1965). "Maximum likelihood estimates of linear dynamic systems." AIAA Journal 3(8), 1445-1450. doi:10.2514/3.3166
6. Schweppe, F. C. (1965). "Evaluation of likelihood functions for Gaussian signals." IEEE Trans. Information Theory 11(1), 61-70. doi:10.1109/TIT.1965.1053737
7. Harvey, A. C., & Phillips, G. D. A. (1979). "Maximum likelihood estimation of regression models with autoregressive-moving average disturbances." Biometrika 66(1), 49-58. doi:10.1093/biomet/66.1.49
8. Jones, R. H. (1980). "Maximum Likelihood Fitting of ARMA Models to Time Series With Missing Observations." Technometrics 22(3), 389-395. doi:10.1080/00401706.1980.10486171
9. Harvey, A. C. (1989). Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge University Press. doi:10.1017/CBO9781107049994
10. Hyndman, R. J., Koehler, A. B., Snyder, R. D., & Grose, S. (2002). "A state space framework for automatic forecasting using exponential smoothing methods." IJF 18(3), 439-454. doi:10.1016/S0169-2070(01)00110-8
11. Durbin, J., & Koopman, S. J. (2012). Time Series Analysis by State Space Methods (2^nd ed.). Oxford University Press. doi:10.1093/acprof:oso/9780199641178.001.0001
12. Julier, S. J., & Uhlmann, J. K. (2004). "Unscented Filtering and Nonlinear Estimation." Proc. IEEE 92(3), 401-422. doi:10.1109/JPROC.2003.823141
13. Gustafsson, F., & Hendeby, G. (2012). "Some Relations Between Extended and Unscented Kalman Filters." IEEE Trans. Signal Processing 60(2), 545-555. doi:10.1109/TSP.2011.2172431
14. Gordon, N. J., Salmond, D. J., & Smith, A. F. M. (1993). "Novel approach to nonlinear/non-Gaussian Bayesian state estimation." IEE Proceedings F 140(2), 107-113. doi:10.1049/ip-f-2.1993.0015
15. Snyder, C., Bengtsson, T., Bickel, P., & Anderson, J. (2008). "Obstacles to High-Dimensional Particle Filtering." Monthly Weather Review 136(12), 4629-4640. doi:10.1175/2008MWR2529.1
16. Särkkä, S., & García-Fernández, Á. F. (2021). "Temporal Parallelization of Bayesian Smoothers." IEEE Trans. Automatic Control 66(1), 299-306. doi:10.1109/TAC.2020.2976316
17. Gu, A., Goel, K., & Ré, C. (2022). "Efficiently Modeling Long Sequences with Structured State Spaces." ICLR. arXiv:2111.00396
18. Gu, A., & Dao, T. (2023). "Mamba: Linear-Time Sequence Modeling with Selective State Spaces." arXiv:2312.00752

Free to read, no ads, no sign-up. If it was useful you can buy me a coffee.