Betelgeuse reaches 24 tok/s for Qwen 3.8 Flash Next on dual-channel RAM
In a self-published r/LocalLLaMA report, the Betelgeuse developer tested Qwen 3.8 Flash Next IQ4_XS on an RTX 3060 12GB with 16GB of system RAM. Betelgeuse reached 24.0 tok/s with dual-channel 2x8GB DDR4 and 14.0 tok/s with a single 16GB stick, while the developer reported byte-exact output and no dropped experts; stock llama.cpp and Strata were slower in the same tests.
The controlled memory-channel comparison suggests that host RAM bandwidth is a meaningful constraint for this Betelgeuse inference setup, but the results are developer-reported and have not received independent output-parity verification.
In my last thread on Betelgeuse (bel-cli / bel-server), the main pushback in the comments was:
- "Okay cool, 22 tok/s on
IQ2_XS, but 2-bit is too low—what happens on an actual 4-bit quant likeIQ4_XSwhere the experts are almost 2x larger?" - "Why were you testing on a single 16GB stick of DDR4 instead of normal dual-channel
2x 8GB?"
Fair questions. So I upgraded the oracle + victim-cache DMA path, packed IQ4_XS (~100GB+ class, full 4-bit experts), and tested it on the exact same RTX 3060 12GB under both RAM configs (still 16GB total system RAM in both tests):
- Config A:
1x 16GBDDR4 (single-channel, ~19 GB/s host bandwidth) - Config B:
2x 8GBDDR4 (normal dual-channel 16GB gaming PC setup, ~38 GB/s host bandwidth)
Measured Results on IQ4_XS (RTX 3060 12GB + 16GB Total RAM)
| Model / Quant | RAM Config (16GB Total) | Stock llama.cpp | Strata (--low-ram) | Betelgeuse (bel-server) | Output Parity vs. Stock |
|---|---|---|---|---|---|
IQ4_XS (4-bit) | 1x 16GB (Single-Channel) | ~0.5 tok/s (thrash) | ~4–5 tok/s | 14.0 tok/s | 100% Byte-Exact |
IQ4_XS (4-bit) | 2x 8GB (Dual-Channel) | ~0.6 tok/s (thrash) | ~5–6 tok/s | 24.0 tok/s | 100% Byte-Exact |
IQ2_XS (GSQ-RCO) | 1x 16GB (Single-Channel) | 1.47 tok/s | 8.11 tok/s | 22.03 tok/s (48 tok/s bursts) | 100% Byte-Exact |
Q8_K_XL (8-bit) | 1x 16GB (Single-Channel) | OOM / <0.1 tok/s | Does not fit | 8.0 cold / 12.0 warm tok/s | 100% Byte-Exact |