DeltaNet

Headlines · 2

1. Oct 23
   Reddit debate asks how AI memory should handle changed information

   A Reddit user argues that long-context AI systems need to update or supersede outdated information, not merely retain and retrieve more. The post uses a hypothetical coding agent with changed API documentation to illustrate conflict resolution, then discusses Transformer, RAG, DeltaNet, and BDH approaches and asks for benchmarks that test response changes under contradictory evidence.

   r/MachineLearning · 🔥 0
2. Oct 18
   KronLSTM applies the Kronecker Delta Rule to long-context linear attention

   An arXiv paper posted on 19 October, "Kronecker Delta Rule for Long Context Linear Attention" by Anamaria-Roberta Hartl, Levente Zólyomi, Michael List, Günter Klambauer, Sebastian Böck and Sepp Hochreiter, proposes KronLSTM, a linear attention architecture built on the Kronecker Delta Rule, described as a new update rule for associative matrix memories. The authors say KronLSTM extends the xLSTM family with a memory-revision mechanism that corrects targeted associations by modifying memory along a single direction in matrix space, reducing interference with stored information; they add that memory dynamics can be described as a Markov chain converging to a steady state, with the expected memory-matrix norm uniformly bounded without extra normalization. The paper reports KronLSTM is competitive with, and often outperforms, existing linear attention architectures including the (Gated) DeltaNet family, especially on long-context benchmarks, and says code and models are open-source.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my DeltaNet experienceAsk about DeltaNet