Betelgeuse在双通道内存上达到24 tok/s
AI 总结
Betelgeuse开发者在r/LocalLLaMA发布自行测试结果：在配备RTX 3060 12GB和16GB系统内存的机器上运行Qwen 3.8 Flash Next IQ4_XS，2x8GB双通道DDR4达到24.0 tok/s，单条16GB内存为14.0 tok/s。开发者还声称没有丢弃专家且输出逐字节一致；同一测试中的stock llama.cpp和Strata速度更慢。
深度见解
这组内存通道对比表明，Betelgeuse推理方案的宿主机内存带宽可能影响MoE模型的运行速度，但测试由开发者自行发布，尚无独立的输出一致性验证。
In my last thread on Betelgeuse (bel-cli / bel-server), the main pushback in the comments was:
- "Okay cool, 22 tok/s on
IQ2_XS, but 2-bit is too low—what happens on an actual 4-bit quant likeIQ4_XSwhere the experts are almost 2x larger?" - "Why were you testing on a single 16GB stick of DDR4 instead of normal dual-channel
2x 8GB?"
Fair questions. So I upgraded the oracle + victim-cache DMA path, packed IQ4_XS (~100GB+ class, full 4-bit experts), and tested it on the exact same RTX 3060 12GB under both RAM configs (still 16GB total system RAM in both tests):
- Config A:
1x 16GBDDR4 (single-channel, ~19 GB/s host bandwidth) - Config B:
2x 8GBDDR4 (normal dual-channel 16GB gaming PC setup, ~38 GB/s host bandwidth)
Measured Results on IQ4_XS (RTX 3060 12GB + 16GB Total RAM)
| Model / Quant | RAM Config (16GB Total) | Stock llama.cpp | Strata (--low-ram) | Betelgeuse (bel-server) | Output Parity vs. Stock |
|---|---|---|---|---|---|
IQ4_XS (4-bit) | 1x 16GB (Single-Channel) | ~0.5 tok/s (thrash) | ~4–5 tok/s | 14.0 tok/s | 100% Byte-Exact |
IQ4_XS (4-bit) | 2x 8GB (Dual-Channel) | ~0.6 tok/s (thrash) | ~5–6 tok/s | 24.0 tok/s | 100% Byte-Exact |
IQ2_XS (GSQ-RCO) | 1x 16GB (Single-Channel) | 1.47 tok/s | 8.11 tok/s | 22.03 tok/s (48 tok/s bursts) | 100% Byte-Exact |
Q8_K_XL (8-bit) | 1x 16GB (Single-Channel) | OOM / <0.1 tok/s | Does not fit | 8.0 cold / 12.0 warm tok/s | 100% Byte-Exact |