DAMP

Headlines · 1

1. Oct 20
   PACE cuts linear-attention model memory with four-bit recurrent states

   Patrick Haller, Filipe Laitenberger and Alan Akbik introduced PACE, a post-hoc quantizer that stores recurrent states at four bits for linear-attention language models. The authors say PACE rounds states only every few generated tokens, shares scales within columns, and retains eight-bit precision for error-prone heads. On four hybrid models ranging from 4B to 35B parameters, the authors report that mixed four- and eight-bit PACE is competitive with eight-bit DAMP on the three larger models while using about two-thirds as much memory.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my DAMP experienceAsk about DAMP