HardenedLinux releases SonicBoom v0.0.3, a compiler-first inference runtime
The HardenedLinux community announced SonicBoom v0.0.3, an open-source GPLv3 inference runtime that it describes as compiler-first and free of Python at build or run time, using one shared library, a C ABI and Guile Scheme as its first-class language. On a single RTX 3050 8GB, the team self-reported that Gemma 4 E4B Q3_K_M reached 1.18× llama.cpp's decode speed and 1.00× prefill, while Qwen2.5-0.5B Q4_K_M reached 1.12× and 1.09×; Qwen decode at depth 512 was 0.98×. The team acknowledges it tested only one GPU and is asking NVIDIA owners to run a one-command benchmark and post results.
The project claims decode speedups over llama.cpp on a single desktop GPU while acknowledging limited testing, and it is publicly asking NVIDIA users to reproduce the numbers.
Hi folks!
We are HardenedLinux, an old-school free software community. SonicBoom is our open-source (GPLv3) inference runtime: a compiler-first alternative to the usual Python stack. No Python at build or run time — one shared library, a C ABI, and Guile Scheme as the first-class language.
What v0.0.3 does on a desktop RTX 3050 8 GB (same session, same GGUF, vs llama.cpp -ngl 99):
| model | decode (tg64) | prefill (pp512) |
|---|---|---|
| Gemma 4 E4B Q3_K_M | 1.18× | 1.00× |
| Qwen2.5-0.5B Q4_K_M | 1.12× | 1.09× |
On real multi-turn agent workloads, time-to-first-token dropped several-fold since v0.0.2 (prefix reuse + plan caching). Output tokens are bit-identical to our golden runs, and prefix reuse is bitwise identical to a full prefill.
How: all graph optimization and kernel generation happen on Boom IR, an MLIR dialect; hot quantized matmuls run as attributed ggml-derived kernels. Every optimization has a switch and an ablation number, so you can see what each one buys.
#HardenedLinux#SonicBoom#llama.cpp#Guile Scheme#MLIR#Gemma 4 E4B#Qwen2.5-0.5B#RTX 3050