Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Kapitsa: open-source Apple Silicon inference engine tuned for Qwen and Nemotron

AI summary

Developer Ye at Nace.ai posted Kapitsa on r/LocalLLaMA, an Apache 2.0 open-source inference engine for Apple Silicon aimed at running coding agents on local models on Macs. Kapitsa supports only two model families — Qwen (3.5 4B, 3.8 27B, 3.6 35B-A3B and 3.8 Flash-Next) and NVIDIA Nemotron (3.5 Lightning, Bolt, Super-VL plus Dory 4B) — so the team can customize Metal kernels per architecture. The author published decode-speed tables on M1 Pro 16GB, M4 Pro 48GB and M5 Max 128GB machines, claiming it beats mlx-serve, oMLX, Ollama, MLX and llama.cpp in most cases, while trailing mlx-serve and oMLX by roughly 2-3% on Qwen3.8 Flash-Next. He also warns readers not to trust benchmarks, points them to a built-in kapitsa bench command to test themselves, notes that 4-bit quantization costs some quality, and says the new codebase should be expected to have bugs.

In-depth insight

The author says the motivation is enterprise networks with no internet access and outdated cloud models, so coding agents must run on local Macs; all performance figures are self-reported and the author himself cautions against trusting them.

Some source content is omitted. Continue reading on the original site.

#Kapitsa#Nace.ai#Qwen#Nemotron#Apple Silicon#MLX#llama.cpp#Ollama#mlx-serve#oMLX

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.