metric

Feature Staleness

The age of the feature values a model scores against, and the divergence between how they are computed at training time and at inference time.

A model trained on features computed one way and served features computed another will underperform in a manner that is hard to diagnose, because the model is correct and the inputs are not what it learned on. This is training-serving skew, and staleness is one of its two causes.

The first cause is age. A feature such as "transactions in the last hour" recomputed nightly is up to a day out of date at inference, while the model was trained on values that were accurate as of the event. The model's reliance on that feature is therefore misplaced in production.

The second is computation. The same feature calculated by a batch SQL job for training and by application code for serving will differ at the edges — timezone handling, null treatment, window boundaries — and the differences are exactly the kind that never appear in a unit test.

The architectural answer is a shared definition executed by one implementation for both paths, which is the core justification for a feature store. Where that is not available, the fallback is to compute training features by replaying the serving path over history rather than by reimplementing it, which preserves the property that matters.