ik_llama.cpp adds native PQ2_0 support for Ternary Bonsai 2
A community guide reports that ik_llama.cpp now provides native CPU and CUDA support for PrismML’s Ternary Bonsai 2 in PQ2_0 format. It cites merged CUDA kernels, a lossless repacking script for upstream TQ2_0 and TQ1_0 weights, and runtime repacking with `-rtr 1` for faster CPU prompt processing; no independent benchmarks are provided.
The reported update gives Ternary Bonsai 2 users a native route for PQ2_0 GPU offload and CPU inference outside PrismML’s own fork.
Hey folks,
Quick heads up for anyone playing around with PrismML's Ternary Bonsai 2!
Until very recently, running it outside of Prism's own fork was a bit of a pain. Upstream llama.cpp uses a block size of QK=256 (TQ2_0), while Prism uses QK=128 (PQ2_0), and trying to offload layers to CUDA on other forks would bottleneck or break due to Hadamard sign tensor handling.
With the latest commits landed in ik_llama.cpp, we finally have complete, native support across both CUDA and CPU. Here is how to set it up cleanly.
- Convert upstream weights losslessly (PR #2694)
If you already have a native PQ2_0 GGUF downloaded from Prism, you can skip this.
#Ternary Bonsai 2#ik_llama.cpp#PrismML#llama.cpp#PQ2_0#TQ2_0#TQ1_0#NVIDIA CUDA