OpenBonsai opens BF16-to-ternary GGUF compression pipeline
The OpenBonsai developer announced an Apache-2.0 framework for converting BF16 models and fine-tunes into ternary GGUF files compatible with Prism's PTQ1_0 format. The developer reports that Qwen3.5-9B was compressed from about 18 GB to 1.997 GB, with the pipeline also applied to Qwen3-8B at 1.8 GB. Its initial models scored below Prism's Ternary Bonsai 2 on most reported reasoning and coding tests, and the developer describes the project as a research preview.
OpenBonsai makes the previously unavailable BF16-to-ternary conversion workflow reproducible, including its calibration, quantization, evaluation scripts and failed experiments, while its results remain developer-reported and show a quality gap versus Prism.
I've been experimenting with tiny models for WebBrain, my open-source browser agent. The long-term goal is to run as much intelligence as possible locally, ideally through WebGPU.
One approach I've been particularly interested in is Prism ML's Ternary Bonsai: compressing a 9B model to roughly 2 GB using ternary weights (-1, 0, +1).
Prism released the weights and inference runtime, but not the complete BF16-to-ternary compression pipeline. You can fine-tune their existing ternary models, but there's no publicly available way to compress your own BF16 fine-tune into their format.
So I tried building one.
Introducing OpenBonsai, an Apache-2.0 framework for post-training ternary quantization, compatible with Prism's PTQ1_0 GGUF format.
How it works
A recent Intel Labs China paper, "Attend to Your Own Thoughts" (Wang et al.), inspired part of the approach: using the model's own reasoning traces for calibration rather than relying exclusively on generic text.
#OpenBonsai#Prism ML#Ternary Bonsai 2#Qwen3.5-9B#Qwen3-8B#PTQ1_0 GGUF#llama.cpp#Intel Labs China