Model Architecture
26 min
Titans: The Sequence Architecture That Learns to Remember While It Runs
Most long-context models compress the past into a fixed-size vector and hope nothing important got squeezed out. Titans instead gives the model a small neural network as its memory, and lets that network keep training on the data…