Nace.AI

Headlines · 2

1. Oct 21
   Kapitsa 0.2 brings KFlash speculative decoding to Apple Silicon, tops self-run benchmark

   Kapitsa, an open-source inference engine for Apple Silicon aimed at Qwen and Nemotron models, released version 0.2 with a new speculative decoding method its developer calls KFlash, announced in a r/LocalLLaMA post. KFlash checks a best-first tree of the draft model's candidate tokens in one pass, whereas the comparison DFlash2 method checks a single chain and stops at the first wrong guess. In the developer's own benchmarks on a MacBook Pro M4 Pro with 48GB, Kapitsa reported 63 tokens/s on Qwen3.8 27B versus 53 tokens/s for Splash 1.4.0 with DFlash2, and 95 versus 62 tokens/s on Qwen3.5 4B on an M1 Pro 16GB machine; the author cautioned readers not to trust benchmarks and to run their own.

   r/LocalLLaMA · 🔥 0
2. Oct 19
   Kapitsa: open-source Apple Silicon inference engine tuned for Qwen and Nemotron

   Developer Ye at Nace.ai posted Kapitsa on r/LocalLLaMA, an Apache 2.0 open-source inference engine for Apple Silicon aimed at running coding agents on local models on Macs. Kapitsa supports only two model families — Qwen (3.5 4B, 3.8 27B, 3.6 35B-A3B and 3.8 Flash-Next) and NVIDIA Nemotron (3.5 Lightning, Bolt, Super-VL plus Dory 4B) — so the team can customize Metal kernels per architecture. The author published decode-speed tables on M1 Pro 16GB, M4 Pro 48GB and M5 Max 128GB machines, claiming it beats mlx-serve, oMLX, Ollama, MLX and llama.cpp in most cases, while trailing mlx-serve and oMLX by roughly 2-3% on Qwen3.8 Flash-Next. He also warns readers not to trust benchmarks, points them to a built-in kapitsa bench command to test themselves, notes that 4-bit quantization costs some quality, and says the new codebase should be expected to have bugs.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my Nace.AI experienceAsk about Nace.AI