RossiLH

Headlines · 1

1. 7d
   Interactive Demo Explores Kimi K3's KDA Attention Module

   RossiLH published a runnable Gradio demo that presents a lightweight reproduction of the Kimi Delta Attention module in Kimi K3. It covers ShortConv, SiTU-GLU, recurrent updates, bounded decay, chunked parallelism, KCP prefix scanning, a complete KimiDeltaAttention layer, and the stated 3:1 KDA-to-MLA scheduling through 90 test functions across 15 tabs. The project runs on CPU and uses reduced or approximated components when no open-source implementation is available.

   掘金 · 人工智能 · 🔥 0

Experience and discussion from the community

Share my RossiLH experienceAsk about RossiLH