Kronecker Delta Rule
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- KronLSTM applies the Kronecker Delta Rule to long-context linear attention
An arXiv paper posted on 19 October, "Kronecker Delta Rule for Long Context Linear Attention" by Anamaria-Roberta Hartl, Levente Zólyomi, Michael List, Günter Klambauer, Sebastian Böck and Sepp Hochreiter, proposes KronLSTM, a linear attention architecture built on the Kronecker Delta Rule, described as a new update rule for associative matrix memories. The authors say KronLSTM extends the xLSTM family with a memory-revision mechanism that corrects targeted associations by modifying memory along a single direction in matrix space, reducing interference with stored information; they add that memory dynamics can be described as a Markov chain converging to a steady state, with the expected memory-matrix norm uniformly bounded without extra normalization. The paper reports KronLSTM is competitive with, and often outperforms, existing linear attention architectures including the (Gated) DeltaNet family, especially on long-context benchmarks, and says code and models are open-source.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my Kronecker Delta Rule experienceAsk about Kronecker Delta Rule
Nobody has shared their experience with Kronecker Delta Rule yet.