Tagged “recurrent-models”

2 posts.

All Model Architecture13 Training & Alignment8 Inference & Serving11 Agents & Orchestration10 Reasoning & Evaluation7 Safety, Security & Governance3 Platforms & Practice11

Model Architecture 15 min

Orthogonalizing Memory Reads: A Muon Trick for Noisy Recurrent Recall

A short experimental note claims that orthogonalizing an mLSTM's matrix memory at read time, borrowing Muon's Newton-Schulz step, sharpens noisy associative recall exactly where the baseline is failing. We walk through the mechan…

recurrent-models mlstm xlstm associative-recall ∑ ◫

Model Architecture 26 min

Titans: The Sequence Architecture That Learns to Remember While It Runs

Most long-context models compress the past into a fixed-size vector and hope nothing important got squeezed out. Titans instead gives the model a small neural network as its memory, and lets that network keep training on the data…

architecture long-context memory test-time-training ∑ ◫