MLA

Headlines · 3

1. 6d
   How KDA Changes Caching for Kimi K3

   A Juejin AI source-code analysis examines how Kimi K3 combines token-addressable MLA history with KDA convolution and recurrent state. It argues that prefix caching must restore a coherent set of MLA, convolution, and recurrent objects at the same boundary rather than merely reuse KV blocks. The article compares SGLang’s KDA-aware checkpointing and unified-memory approach with vLLM’s state blocks, metadata, fine-grained partial hits, and retention strategies.

   掘金 · 人工智能 · 🔥 0
2. Nov 3
   Trade-offs in Distributed Inference for Attention and MoE

   The report analyzes distributed-inference strategies across tensor, data, expert, context, and pipeline parallelism using weight, KV-cache, activation, communication, capacity, and latency budgets. Its author argues that GQA can shard KV heads up to the KV-head count, while MLA's shared latent cache must be replicated under tensor parallelism, and compares MoE tensor parallelism with expert parallelism in terms of communication, memory use, load balancing, and implementation complexity. The report title labels it installment 2, while the body calls it the third of eight installments in an inference-infrastructure methodology series. Its comparative figures and conclusions are the author's analysis and are not independently corroborated by the supplied evidence.

   掘金 · 人工智能 · 🔥 0
3. Oct 23
   Virtual Head Attention targets long-context LLM efficiency

   The r/LocalLLaMA post introduces Virtual Head Attention (VHA), an attention method using Q Premix and Linear Postmix to address tradeoffs among KV-cache size, compute, and quality. The authors report that VHA can be folded into a standard GQA-2 architecture for inference, beat a GQA baseline across four pretraining scales, reduce KV cache by 4× versus GQA-8 at dense scales and 2× versus GQA-4 at the MoE scale, and lower training and prefill FLOPs by 7.9% in a reported 1.7B, 4K-context run.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my MLA experienceAsk about MLA

All

annotated_deep_learning_paper_implementa•4d Pinned

annotated_deep_learning_paper_implementations: Annotated PyTorch implementations and tutorials for understanding deep learning algorithms.

What problem it solves This project provides simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations and rendered as side-by-side formatted notes to help readers understand the algorithms better. Key features PyTorch Implementations — The project provides implementations of neural networks and related algorithms using PyTorch…

github.comhttps://github.com/labmlai/annotated_deep_learning_paper_implementations