Sequential Editing and Model Collapse
Editing a model once is cheap and safe; editing it a thousand times in sequence produces first gradual forgetting and then an abrupt loss of general ability, which is the single biggest obstacle to using weight editing as a maintenance mechanism.
A single rank-one edit is nearly free and, measured on the edited fact, works. The interesting regime is the one a production team would actually be in: a stream of corrections arriving over months, applied one after another to the same weights. That regime behaves qualitatively differently.
Editing ROME and MEMIT sequentially produces two phases. First a gradual phase, where previously edited facts start to be forgotten and downstream benchmark scores drift down slowly. Then an abrupt phase, where the model loses general ability in a way no amount of further editing repairs. For MEMIT the catastrophic phase has been reported to begin at roughly 1,400 sequential edits, with the gradual phase lasting considerably longer than ROME's (Gupta et al., 2024, Model Editing at Scale leads to Gradual and Catastrophic Forgetting, ACL Findings 2024, arXiv:2401.07453).
Why the weights drift
Each edit adds a rank-one term to the same matrix. After \(n\) edits the deviation from the original weights is a sum of \(n\) outer products:
If the edit directions were mutually orthogonal and the model's other computations lived in the orthogonal complement, this would be harmless. Neither holds. Key directions for real facts are correlated, the residual stream is a shared bus rather than a set of private channels, and each edit's preservation term is computed against the original key covariance rather than against the state left behind by the previous edits. Errors therefore compound rather than cancel, and the norm of the accumulated perturbation grows until it is comparable to the singular values of the matrix being edited. Past that point the layer is no longer doing its original job.
Norm growth is the practical early-warning signal. Implementations that produce large, spiky parameter changes collapse early; implementations whose changes stay small and grow smoothly survive far longer.
Disabling edits, and how much was a bug
Some individual edits cause immediate collapse rather than gradual decay. These "disabling edits" were traced in part to irregularities in the original ROME implementation: a corrected version, r-ROME, produces parameter changes that are orders of magnitude smaller and increase smoothly, and the collapse was observed with the CounterFact dataset but not with zsRE (Gupta, Baskaran & Anumanchipalli, 2024, Rebuilding ROME, arXiv:2403.07175).
This is the honest complication in the story. Part of what looked like a fundamental limit of rank-one editing was an artefact, and better implementations push the collapse point out. Part of it is not: even r-ROME and methods designed explicitly for high edit volumes converge back toward pre-edit accuracy when evaluated over tens of thousands of real updates (Thede et al., 2025, WikiBigEdit, ICML 2025, arXiv:2503.05683).
Constraining the update instead of shrinking it
The most effective mitigation so far is geometric. AlphaEdit projects each candidate perturbation onto the null space of the preserved knowledge's key matrix before applying it, so the update is orthogonal to the directions in which retained facts are represented. Reported across GPT2-XL, GPT-J and LLaMA3, this raises the performance of locating-then-editing methods by an average of about 36 percent (36.4 in one version of the paper, 36.7 in a later one) (Fang et al., 2025, AlphaEdit, ICLR 2025 outstanding paper, arXiv:2410.02355). Related work adds orthogonality constraints between successive edits so that later edits avoid the subspaces earlier ones used.
Null-space projection buys headroom; it does not change the shape of the problem. The null space has finite dimension, and every edit consumes some of it.
When it breaks
Batched and sequential are not the same experiment. Applying 10,000 edits in one solve is a different numerical problem from applying them one at a time, and results from the first do not transfer to the second. Papers that report large batch sizes are often silent about the sequential case.
Perplexity hides it. A model can hold fluency while losing factual recall and instruction-following. Collapse is visible earlier in downstream task accuracy than in a language-modelling loss.
Rollback is not free. Weight edits are destructive and cumulative. Unless you keep the base checkpoint plus an ordered log of edits, and can replay them, "undo the third edit" is not an operation you have.
6 flashcards for this concept
Click a card to reveal the answer.