EngramEdit edits facts in LLMs through conditional memory
A new arXiv paper, EngramEdit, builds on conditional memory architectures such as DeepSeek Engram — which look up learned embeddings from input n-grams to expand model capacity at limited extra computation — to decouple factual knowledge storage from general computation and update facts while keeping the Transformer backbone fixed. It first computes target memory representations that make the model predict the updated fact across multiple expressions, then jointly updates the shared n-gram embeddings, penalizing changes to frequently reused embeddings more strongly to preserve unrelated knowledge. The authors report near-perfect editing success, generalization of revised knowledge to unseen expressions and multi-hop reasoning, roughly three times the strongest baseline's chain-of-thought accuracy, and largely preserved unrelated knowledge and general capabilities as edits accumulate.
If reproducible, an editable conditional-memory layer offers a route to continuously update factual knowledge without retraining the model.
Computer Science > Computation and Language
arXiv:2610.10533 (cs)
[Submitted on 7 Oct 2026]
Title:EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
Authors:Hongru Cai, Ran Wei, Wenjie Wang, Chengfa Wu, Ning Song, Yongqi Li, Wenjie Li
View a PDF of the paper titled EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory, by Hongru Cai and 6 other authors