Memory-Based and Non-Parametric Editing

If the expensive part of weight editing is that it is destructive, cumulative and hard to reverse, the obvious alternative is not to touch the weights. Keep the edits somewhere else, decide at inference time whether the current input falls inside the scope of any of them, and if it does, answer from the store instead of from the parameters.

That single design idea covers several methods that look different on paper.

The family

SERAC keeps an explicit edit memory, a scope classifier that decides whether an input is affected by any stored edit, and a small counterfactual model that answers when it is; the base model is never modified. The edit memory itself is tiny, on the order of a few kilobytes per edit, with most of the additional footprint coming from the classifier and counterfactual model (Mitchell et al., 2022, Memory-Based Model Editing at Scale, ICML 2022).

GRACE keeps a discrete codebook and overwrites intermediate activations at inference when a key matches, which makes it a reliable way to remember many edits and a poor way to generalise to rephrasings of them (Hartvigsen et al., Aging with GRACE, arXiv:2211.11031).

WISE adds routing and memory sharding so that edits land in separate side memories and interfere with each other less (Wang et al., 2024, WISE, arXiv:2405.14768).

In-context editing is the degenerate case with no machinery at all: retrieve the relevant new facts and put them in the prompt. On ripple-effect evaluation this baseline outperformed the parametric methods it was meant to be a strawman for (Cohen et al., 2024, TACL).

Why it generalises better

A fact in the context is visible to every circuit in the forward pass. Composition, comparison and multi-hop reasoning all read the same residual stream, so a retrieved fact participates in reasoning the model already knows how to do. A fact written into one MLP matrix participates only where that key is matched.

The cost moves rather than disappearing. Every request now pays a retrieval step plus the tokens the fact occupies, and prefill grows with the number of facts injected. Weight editing pays once at edit time and nothing per request; memory-based editing pays nothing at edit time and something on every request forever. Which is cheaper depends on traffic, not on elegance.

Scope is the hard part

The difficult component is not the store, it is deciding when an input is in scope. Fire too eagerly and the model is overridden on questions the edit should not touch; fire too conservatively and the edit is invisible. SERAC learns this as an explicit classifier. GRACE decides by activation-space proximity, with a radius per codebook entry. A retrieval pipeline decides it by similarity threshold and reranking, which is the same problem wearing a different hat.

Scope errors are also the failure mode users notice, because an over-eager edit produces a confidently wrong answer on an adjacent question rather than a stale one.

When it breaks

Latency and cost scale with the edit set. A thousand edits is a retrieval index, not a lookup table, and it needs the same engineering as any other retrieval system: embeddings, reranking, freshness, evaluation.

Multi-hop is still hard. Retrieval-augmented approaches lead lifelong editing benchmarks on nearly every axis except multi-hop reasoning, where composition over several retrieved edits remains weak (Thede et al., 2025, WikiBigEdit, ICML 2025).

The base model still disagrees. A retrieved fact competes with a memorised one, and which wins depends on the model, the prompt and how confidently the parametric fact was learned. Instruction wording that privileges retrieved context is doing real work.

Nothing gets cheaper at inference. If your reason for editing was to avoid paying context tokens for a correction on every request, memory-based editing does not give you that.