Tagged “recurrent-models”

3 posts.

All Model Architecture19 Training & Alignment22 Inference & Serving18 Agents & Orchestration11 Reasoning & Evaluation26 Safety, Security & Governance7 Platforms & Practice20

Model Architecture 15 min

Orthogonalizing Memory Reads: A Muon Trick for Noisy Recurrent Recall

A short experimental note claims that orthogonalizing an mLSTM's matrix memory at read time, borrowing Muon's Newton-Schulz step, sharpens noisy associative recall exactly where the baseline is failing. We walk through the mechan…

recurrent-models mlstm xlstm associative-recall ∑ ◫

Model Architecture 24 min

The Kalman Filter: Sixty-Six Years of Bayes' Rule, One Observation at a Time

In the fall of 1960 Rudolf Kalman presented a paper at NASA Ames that engineers found hard to grasp; by early 1961 it was navigating simulated spacecraft around the Moon. The same recursion now computes the exact likelihood of ev…

time-series-foundations statistics forecasting state-space-models ∑ ◫

Model Architecture 26 min

Titans: The Sequence Architecture That Learns to Remember While It Runs

Most long-context models compress the past into a fixed-size vector and hope nothing important got squeezed out. Titans instead gives the model a small neural network as its memory, and lets that network keep training on the data…

architecture long-context memory test-time-training ∑ ◫