Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Muir linear-attention model targets long-context recall and streaming

AI summary

The Muir paper introduces a linear-attention architecture that combines sensitivity-allocated gating, matched-filter readout, and a numerically stable parallel update. Its authors report lower perplexity and stronger recall than Gated DeltaNet-2 and Transformer baselines, along with 10M-token streaming from a 4K training length and near-perfect passkey retrieval up to 1M tokens. A Reddit analysis also describes a 25 MB state at 10M tokens, but these technical and benchmark claims have not been independently confirmed.

In-depth insight

The paper targets a central limitation of recurrent and linear-attention models: retaining useful information over very long contexts without letting slow-decaying state rows or unstable training kernels dominate behavior.

Computer Science > Machine Learning

arXiv:2611.08376 (cs)

[Submitted on 5 Nov 2026]

Title:Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout

Authors:Tushaar Gangavarapu, Gil Tabak, Clemens JS Schaefer

View a PDF of the paper titled Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout, by Tushaar Gangavarapu and 2 other authors

#Muir#Gated DeltaNet-2#Transformer

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.