Latent Compaction
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Latent Compaction compresses KV caches via test-time training
A new arXiv paper introduces Latent Compaction, a method that learns a compact, document-specific KV cache through test-time training to cut the memory and attention cost of long-context generation. Cache entries are initialized using reconstruction importance, teacher attention and key-space diversity; training first fits attention outputs and log-normalizers to those of the full cache, then jointly optimizes the whole cache via end-to-end distillation from the full-cache model, with model weights frozen and only generic document prompts used, not downstream questions or answers. The authors report gains over budget-matched compression baselines on RULER, LongBench and Phonebook Lookup — for example 96.7% accuracy at 32x compression on RULER versus 40.0% for KVZip+ — and say the method transfers across model scales and families, improving retrieval on Qwen3 and MiniCPM5. Code is released.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my Latent Compaction experienceAsk about Latent Compaction
Nobody has shared their experience with Latent Compaction yet.