Time Series Foundation Models

The pretraining recipe that worked for language has been applied to time series: train a large sequence model on a broad corpus of series from many domains, then forecast a series it has never seen with no fitting at all. Several such models now exist, and the interesting questions are what is being transferred and whether the comparison being reported is a fair one.

Why transfer is plausible

Time series across domains share structure. Trend, seasonality at daily, weekly and annual periods, level shifts, mean reversion, spikes and decay appear in electricity demand, web traffic, retail sales and sensor readings alike. A model that has learned to recognise and continue these patterns has learned something that transfers, in the same way that syntax transfers across text domains.

The architectural choices vary: patch-based encoder-decoders, decoder-only models over tokenised or quantised values, and models predicting distributional parameters. What they share is treating the numeric series as a sequence and pretraining on a large heterogeneous corpus.

What zero-shot forecasting buys

No per-series fitting. For a large catalogue this removes the training pipeline entirely, which is a substantial operational simplification.

Immediate cold start. A new series is forecast from its first few observations rather than after accumulating enough history to fit.

A strong default. For a team without forecasting expertise, a zero-shot model that is competitive with a tuned classical method is worth more than the accuracy comparison alone suggests.

Reading the claims

The comparison that matters is against a well-tuned baseline, and this is where reported evaluations frequently fall short. Seasonal naive, exponential smoothing with automatic parameter selection, and a gradient-boosted tree on lag features are all cheap and are strong, and a foundation model that beats an untuned ARIMA has beaten a straw baseline.

Contamination is the second question. Public benchmark datasets are widely available and may appear in pretraining corpora, so a zero-shot claim on a well-known benchmark needs the pretraining corpus described.

The honest summary as of now is that these models are competitive with tuned classical methods on many series and are not uniformly better, and that their operational advantages are often more compelling than their accuracy.

When it breaks

Covariates are the weak point. Real forecasting depends on promotions, prices, holidays and weather, and support for exogenous variables in these models is more limited than in a gradient-boosted tree where adding a feature is trivial. This is the most common reason a practitioner returns to a supervised approach.

Fine-tuning usually beats zero-shot on a specific domain. Zero-shot is the demonstration; fitting on your own series recovers the accuracy that domain specificity provides, and it reintroduces the training pipeline that zero-shot was meant to remove.

Inference cost is real. A large model called per series per horizon is far more expensive than exponential smoothing, and for half a million series that cost is a serious line item rather than a rounding error.

Intermittent and count data are handled poorly. Series that are mostly zeros with occasional spikes, which describes a large share of retail demand, do not resemble the smooth series that dominate pretraining corpora, and specialised methods still win there.