Local LLM engines benchmarked on RTX 3060 with Qwen3.6-35B-A3B
A Reddit user reported a single-system benchmark comparing RustingStrata, PolyStrata, and llama.cpp b12119 while running Qwen3.6-35B-A3B on an RTX 3060 12 GB. With a 5,290-token prompt, RustingStrata recorded 2,330 tok/s prefill versus 1,458–1,549 tok/s for PolyStrata and 1,692 tok/s for llama.cpp; no-MTP decode was reported at 90.9 tok/s, 78.5 tok/s, and 53.5 tok/s respectively. The comparison used different prompts and configurations, and the report notes that its PolyStrata no-MTP run still accepted some drafts.
The results provide useful, detailed evidence for comparing local-LLM inference engines on consumer hardware, but they come from one independently self-reported Reddit test on a single machine, model, and quantization rather than a broadly corroborated evaluation.
RTX 3060 12 GB, Ryzen 5 7600X, 32 GB RAM, one GGUF file: Qwen3.6-35B-A3B-UD-Q3_K_M.gguf (Unsloth, MTP layer included, 15.9 GiB, does not fit in VRAM). Prompt of 5290 tokens, 128 generated tokens.
| RustingStrata | PolyStrata | llama.cpp b12119 | |
|---|---|---|---|
| Prefill, 5290-token prompt | 2330 tok/s | 1458-1549 tok/s | 1692 tok/s (-ncmoe 22, -ub 2048) |
| Decode, no MTP | 90.9 tok/s | 78.5 tok/s (--no-draft, see below) | 53.5 tok/s |
| Decode with MTP drafts | 128.4 tok/s | 97.5-100.4 tok/s | 77.7 tok/s (--spec-type draft-mtp, 92 of 102 drafts accepted) |
How each was run:
- RustingStrata:
rustingstrata bench --gpu --mtp 0|1 --pp 5290 --tg 128 --reps 3, mean of 3. - PolyStrata: its own
polystrata.py benchmark --model qwen3.6 --prompt-tokens 4000 --max-tokens 128. That prompt came to 5290 tokens, so I used that length for the other two. I ran it three times; the first run after setup was cold (decode 24.6 tok/s) and is left out, the numbers above are runs 2 and 3. MTP drafts are on by default. For the no-MTP row I added the engine's--no-draftflag to its config, it still offered a few drafts afterwards (6 of 16 and 15 of 36 accepted in the log), so that row is not a clean no-MTP number. Prefill in those runs: 1602-1643 tok/s. - llama.cpp:
llama-bench -ngl 99 -fa 1 -ub 2048 -b 2048 -p 5290 -n 128 -r 3, with-ncmoeswept over 18 to 26. 22 was the fastest that fits in 12 GB (no MTP). The MTP row isllama-server --spec-type draft-mtpwith-ncmoe 23(22 does not fit with the MTP context and drops to 20 tok/s), a 5381-token prompt of repo docs, 3 runs, prefill about 1565 tok/s.
#RustingStrata#PolyStrata#llama.cpp#Qwen3.6-35B-A3B#RTX 3060