RustingStrata与llama.cpp的Qwen背靠背测试
一名 Reddit 用户在同一台配备 RTX 3060 12 GB 的电脑上，使用相同的 Qwen GGUF 文件，对 RustingStrata 和 llama.cpp 进行了背靠背测试。对于超出显存容量的大模型，用户报告称 RustingStrata 的预填充速度约为 1.5 倍，解码速度最高约为 2 倍；llama.cpp 在小型稠密模型上的预填充速度更快。这些数据是未调优、桌面程序运行状态下两次测试结果的平均值。
相同电脑和模型文件的对比为评估本地 Qwen 推理引擎提供了有用数据，但结果目前仍来自单个用户，尚未获得独立验证。
Ran every Qwen GGUF on my disk through RustingStrata and llama.cpp, same PC, same file, back to back.
PC: RTX 3060 12 GB, Ryzen 5 7600X, 32 GB RAM. CUDA. Normal desktop open (browser, Discord), no tuning, so numbers are lower than in the README.
| Model | Quant | Size | Prefill 4K, RustingStrata | Prefill 4K, llama.cpp | Decode after 4K, RustingStrata | Decode after 4K, llama.cpp |
|---|---|---|---|---|---|---|
| Qwen3.5-2B | Q8_0 | 1.9 GB | 3859 | 6080 | 96.1 | 89.8 |
| Qwen3-4B-Instruct-2507 | Q4_K_M | 2.3 GB | 1433 | 2482 | 57.4 | 59.7 |
| Qwen3.8-27B (Swift-1.5) | IQ2_XXS | 8.3 GB | 283 | 459 | 16.5 | 14.7 |
| Qwen3-30B-A3B-Instruct-2507 | UD-Q2_K_XL | 11 GB | 1236 | 1545 | 63.6 | 52.9 |
| Qwen3.6-35B-A3B | UD-Q2_K_XL | 11.7 GB | 1539 | 1029 | 71.5 | 39.6 |
| Qwen3-Next-80B-A3B-Instruct | UD-IQ1_S | 21 GB | 1154 | 739 | 63.7 | 33.3 |
| Qwen3.8-Flash-Next (177B) | IQ2_XS | 63 GB | 305 | 193 | 21.3 | 18.1 |
All numbers tok/s, mean of 2 runs.
Short version: small dense models, llama.cpp still prefills faster. Once the model is bigger than VRAM (35B, 80B, 177B on a 12 GB card), Strata is ~1.5x on prefill and up to ~2x on decode. 80B at 64 tok/s and a 177B model at 21 tok/s on a 3060 with 32 GB RAM.
Commands: - Strata: strata bench --model X --gpu --mtp 0 --pp 4096 --tg 128 --reps 2 - llama.cpp (build 12119): llama-bench -m X -p 4096 -n 128 -d 4096 -fa on -ub 2048 -b 2048 -fitt 512 (its auto-fit to VRAM)
MTP is off here. With MTP on, 35B goes up to ~160 tok/s decode.
Hand-tuned llama.cpp (-ncmoe 12) on a quiet machine does better on 35B (~2236 prefill / 68 decode), see the README for that comparison.
Repo: RustingStrata
#RustingStrata#llama.cpp#Qwen3.5-2B#Qwen3-4B-Instruct-2507#Qwen3.8-27B#Qwen3-30B-A3B-Instruct-2507#Qwen3.6-35B-A3B#Qwen3-Next-80B-A3B-Instruct#Qwen3.8-Flash-Next