EngramEdit

Headlines · 1

1. Oct 6
   EngramEdit edits facts in LLMs through conditional memory

   A new arXiv paper, EngramEdit, builds on conditional memory architectures such as DeepSeek Engram — which look up learned embeddings from input n-grams to expand model capacity at limited extra computation — to decouple factual knowledge storage from general computation and update facts while keeping the Transformer backbone fixed. It first computes target memory representations that make the model predict the updated fact across multiple expressions, then jointly updates the shared n-gram embeddings, penalizing changes to frequently reused embeddings more strongly to preserve unrelated knowledge. The authors report near-perfect editing success, generalization of revised knowledge to unseen expressions and multi-hop reasoning, roughly three times the strongest baseline's chain-of-thought accuracy, and largely preserved unrelated knowledge and general capabilities as edits accumulate.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my EngramEdit experienceAsk about EngramEdit