Mamba-2

Headlines · 3

1. 2d
   Developer Ports PyTorch 2.7.1 and Mamba-1 to Legacy NVIDIA GPUs

   A developer announced a Windows port of an LLM training stack for NVIDIA GPUs from SM 3.5 through SM 7.5, including a custom PyTorch 2.7.1 build with CUDA 11.8, ported causal-conv1d, native-CUDA Mamba-1 and Windows NCCL. The author reports that Mamba-1 works across those architectures and achieved a maximum absolute error of 3.8e-06 against a PyTorch reference on a 2013 Tesla K40c. Mamba-2 and Mamba-3 require Turing or newer because they depend on Triton and TileLang, while older GPUs automatically fall back to Mamba-1.

   r/LocalLLaMA · 🔥 0
2. Nov 2
   CPU Reproduction Explores Kimi K3’s Kimi Delta Attention

   A Juejin AI article presents the first part of a lightweight, CPU-only reproduction of Kimi K3’s Kimi Delta Attention, without GPU use or large-scale training. It analyzes state recurrence, delta-rule writes, bounded logarithmic decay, channel-wise forgetting, head-wise RMSNorm, output gating, and comparisons with other attention and state-space mechanisms, alongside runnable implementation code and demos.

   掘金 · 人工智能 · 🔥 0
3. Oct 19
   Jane Street and NVIDIA Release Mamba-2 cuTile Kernels

   Jane Street engineers and NVIDIA implemented three custom kernels for the Mamba-2 forward pass in CUDA Tile (cuTile), a Python-based DSL and parallel programming model for tile-level NVIDIA GPU programming. NVIDIA reports that the kernels were 1.14x to 1.82x faster than an open-source Triton implementation in tests on a single NVIDIA B300 SXM6. The results measure kernel performance only; end-to-end training performance will also depend on backward-pass implementations.

   NVIDIA Technical Blog · 🔥 0

Experience and discussion from the community

Share my Mamba-2 experienceAsk about Mamba-2