Model Architecture
15 min
Orthogonalizing Memory Reads: A Muon Trick for Noisy Recurrent Recall
A short experimental note claims that orthogonalizing an mLSTM's matrix memory at read time, borrowing Muon's Newton-Schulz step, sharpens noisy associative recall exactly where the baseline is failing. We walk through the mechan…