SonicBoom v0.0.3 推理运行时发布
HardenedLinux 社区在 r/LocalLLaMA 发布开源 GPLv3 推理运行时 SonicBoom v0.0.3，称其采用编译器优先架构，构建和运行都不依赖 Python，使用单一共享库、C ABI，并以 Guile Scheme 作为一等语言。该团队在单张 RTX 3050 8GB 上自测：Gemma 4 E4B Q3_K_M 的 decode 为 llama.cpp 的 1.18 倍、prefill 为 1.00 倍；Qwen2.5-0.5B Q4_K_M 分别为 1.12 倍和 1.09 倍，但在深度 512 的 Qwen decode 上为 0.98 倍。团队承认只有一块 GPU，并呼吁 NVIDIA 用户按统一命令提交基准结果，以补足证据。
该项目声称在单张桌面 GPU 上 decode 速度超过 llama.cpp，同时承认测试范围有限，并公开征集 NVIDIA 用户复现结果。
Hi folks!
We are HardenedLinux, an old-school free software community. SonicBoom is our open-source (GPLv3) inference runtime: a compiler-first alternative to the usual Python stack. No Python at build or run time — one shared library, a C ABI, and Guile Scheme as the first-class language.
What v0.0.3 does on a desktop RTX 3050 8 GB (same session, same GGUF, vs llama.cpp -ngl 99):
| model | decode (tg64) | prefill (pp512) |
|---|---|---|
| Gemma 4 E4B Q3_K_M | 1.18× | 1.00× |
| Qwen2.5-0.5B Q4_K_M | 1.12× | 1.09× |
On real multi-turn agent workloads, time-to-first-token dropped several-fold since v0.0.2 (prefix reuse + plan caching). Output tokens are bit-identical to our golden runs, and prefix reuse is bitwise identical to a full prefill.
How: all graph optimization and kernel generation happen on Boom IR, an MLIR dialect; hot quantized matmuls run as attributed ggml-derived kernels. Every optimization has a switch and an ablation number, so you can see what each one buys.
#HardenedLinux#SonicBoom#llama.cpp#Guile Scheme#MLIR#Gemma 4 E4B#Qwen2.5-0.5B#RTX 3050