KronLSTM applies the Kronecker Delta Rule to long-context linear attention
An arXiv paper posted on 19 October, "Kronecker Delta Rule for Long Context Linear Attention" by Anamaria-Roberta Hartl, Levente Zólyomi, Michael List, Günter Klambauer, Sebastian Böck and Sepp Hochreiter, proposes KronLSTM, a linear attention architecture built on the Kronecker Delta Rule, described as a new update rule for associative matrix memories. The authors say KronLSTM extends the xLSTM family with a memory-revision mechanism that corrects targeted associations by modifying memory along a single direction in matrix space, reducing interference with stored information; they add that memory dynamics can be described as a Markov chain converging to a steady state, with the expected memory-matrix norm uniformly bounded without extra normalization. The paper reports KronLSTM is competitive with, and often outperforms, existing linear attention architectures including the (Gated) DeltaNet family, especially on long-context benchmarks, and says code and models are open-source.
The work couples an analytically tractable, stable memory rule to recurrent linear attention; if its self-reported long-context gains hold up, it offers the xLSTM lineage a new route to long-context modeling.
Computer Science > Machine Learning
arXiv:2610.26834 (cs)
[Submitted on 19 Oct 2026]
Title:Kronecker Delta Rule for Long Context Linear Attention
Authors:Anamaria-Roberta Hartl, Levente Zólyomi, Michael List, Günter Klambauer, Sebastian Böck, Sepp Hochreiter
View a PDF of the paper titled Kronecker Delta Rule for Long Context Linear Attention, by Anamaria-Roberta Hartl and 5 other authors
#KronLSTM#xLSTM#Kronecker Delta Rule#DeltaNet#Gated DeltaNet#Sepp Hochreiter#arXiv