Online Training Loop

A nightly batch pipeline fixes the maximum speed at which a model can respond to the world at one cycle. For a credit risk model that is irrelevant. For a recommendation feed, where the interesting content and the users' interest in it both shift within hours, it means every prediction is made with yesterday's beliefs about today's catalogue.

The online training loop removes the cycle. A training worker consumes the interaction stream continuously, updates parameters, and pushes them to serving on a cadence measured in minutes. What disappears along with the cycle is the artefact, and with it the gate: there is no longer a candidate model to evaluate, compare against the incumbent and promote.

Why it matters

The pattern is spreading well beyond recommendations — fraud scoring, ranking, pricing, ad selection — and it is usually adopted for the freshness benefit with the governance consequence discovered later. The served model is now a continuously mutating state, not a versioned object. Answering "which model made this decision" requires deliberate machinery: parameter snapshots, an identifier stamped on every prediction, and retention of both.

It also creates a feedback loop with a direction. The model's own outputs generate the impressions and clicks it subsequently learns from, so an error is amplified rather than corrected: items the model stops showing generate no positive signal, which confirms the model's belief that they are not worth showing.

Implementation patterns

• Snapshot parameters on a fixed cadence — every 5 to 15 minutes in fast-moving systems — so the worst rollback loses minutes of learning rather than everything since the last daily save.
• Keep sparse features in a collisionless table rather than hashing them into a fixed-size array. ByteDance's Monolith paper (RecSys workshop, 2022) makes this the centrepiece: collisions between two ids sharing a slot are tolerable when a model is retrained nightly and corrosive when it is learning continuously, because the corrupted embedding is never rebuilt from scratch.
• Bound memory with expiry and frequency filtering, since a collisionless table grows with every distinct id ever seen. Entries unused for a period are dropped; ids seen fewer than N times never get an entry.
• Validate the training stream, not just the serving metrics. Anomaly detection on input distribution, event rate and label rate catches the bad hour before the model has absorbed it.
• Build a freeze switch that stops parameter updates while serving continues, which is the only fast mitigation available when quality is degrading and the cause is unknown.

Industry example

Monolith, described by ByteDance in 2022, is a production recommendation system built around exactly this trade. The paper's stated position is that system reliability was traded for real-time learning: in their workload, reacting to user feedback within minutes was worth more than the fault tolerance a batch-and-promote pipeline provides. The system pairs collisionless embedding tables with online training and reports the arrangement as production-proven.

Failure scenarios

• Poisoned by a bad hour. Bot traffic, a logging bug or a broken upstream feature is learned rather than quarantined, and the only signal is a business metric drifting a few percent.
• No artefact to roll back to. Without frequent snapshots, recovery means retraining from a checkpoint that may be a day old, during an active quality incident.
• Silent divergence between training and serving features, which in a batch world is caught by offline evaluation and in an online world has no gate at all.
• Runaway memory, when expiry is mistuned and the collisionless table grows without bound until the training worker dies.

Trade-offs

Gains: freshness measured in minutes, adaptation to new items without waiting for a cycle, and no train-serve gap for the most recent behaviour. Pays: loss of the promotion gate, snapshot storage and complexity, a harder regulatory story, a feedback loop that can self-reinforce, and an on-call burden that now includes the training path, which in a batch system could fail overnight without anybody waking up.

When not to use it

When the thing being predicted moves slower than the training cycle. Churn propensity, credit risk, warehouse demand: the world does not change materially in an hour, and the gate you give up is worth more than the freshness you gain. Also when the decision is reviewable or regulated, because "the model that produced this outcome" must be a reproducible artefact, and a continuously mutating parameter set is not one without substantial extra machinery. The deciding measurement is simple and rarely taken: train one model on data to yesterday and one on data to the last hour, and compare. If the gap is small, the nightly batch is the better system, and it is cheaper to run.

Interview question

Q: You run online training on a ranking model. A logging change at 02:00 doubles the click events for one surface. Nobody notices until 11:00. What is the state of your system, and what do you do first?

What a strong answer covers: the model has learned nine hours of a skewed distribution and serving is already affected · freeze parameters first, before diagnosing, because every minute adds more of it · roll back to the snapshot before 02:00 and replay clean data if available · the feedback loop meaning the skew is partly self-sustaining · why the detector should have been on the training stream's event rate rather than on serving quality · what a batch pipeline would have done differently, and the honest admission that it would have caught this at the promotion gate.

Quick check

Quiz: What does an online training loop remove that a batch pipeline relies on? The promotable artefact, and therefore the evaluation gate before traffic sees the model.

Flashcard: Why do hash collisions in embedding tables matter more with online training? — Because nothing ever rebuilds the table from scratch, so a shared slot corrupts both ids' representations indefinitely rather than until the next nightly run.