Interactive Demo Explores Kimi K3's KDA Attention Module
RossiLH published a runnable Gradio demo that presents a lightweight reproduction of the Kimi Delta Attention module in Kimi K3. It covers ShortConv, SiTU-GLU, recurrent updates, bounded decay, chunked parallelism, KCP prefix scanning, a complete KimiDeltaAttention layer, and the stated 3:1 KDA-to-MLA scheduling through 90 test functions across 15 tabs. The project runs on CPU and uses reduced or approximated components when no open-source implementation is available.
The demo gives AI engineers an interactive way to inspect KDA mathematics, prefill and decode consistency, numerical diagnostics, and hybrid layer scheduling without a GPU or large-scale training.
Some source content is omitted. Continue reading on the original site.
#Kimi K3#Kimi Delta Attention#KimiDeltaAttention#Gradio#PyTorch#RossiLH