Reddit user reports Speedy outperforming Avalon and Strata
A Reddit user reported that the Speedy local-LLM inference engine outperformed Avalon on cold prefill, reading speed and total exchange time at 60K, 120K and 198K context lengths. In the same test on an RTX 4090 with 128 GB DDR5-3600 memory and a Ryzen 9 7950X, Speedy reportedly generated tokens about 45% faster than Strata and slightly faster than Avalon. The figures use ISTA IQ3_S quantization and were not independently verified.
The post provides a concrete, configuration-specific benchmark signal for developers evaluating local LLM inference engines at long context lengths, though its results remain an unverified single-user report.
I had just finished setting up avalon, then i found speedy, which crushed avalon.
https://github.com/freriadoc/Avalon https://github.com/DevDock-AI/Speedy
Hardware: 4090, 128gb ddr5 3600, ryzen 9 7950x
ISTA IQ3_S quant:
| Context | Reading: cold prefill s | Reading: reported tok/s | Writing: generation tok/s | Whole exchange: wall s |
|---|---|---|---|---|
| 60K | Strata 54.7 / Avalon 22.4 / Speedy 13.7 | Strata — / Avalon 2678 / Speedy 4389 | Strata 75.0 / Avalon 102.5 / Speedy 110.0 | Strata 66.6 / Avalon 32.1 / Speedy 23.9 |
| 120K | Strata 110.7 / Avalon 42.1 / Speedy 26.0 | Strata — / Avalon 2849 / Speedy 4622 | Strata 71.8 / Avalon 101.6 / Speedy 105.8 | Strata 123.2 / Avalon 52.0 / Speedy 37.0 |
| 198K | Strata 187.2 / Avalon 71.2 / Speedy 44.1 | Strata — / Avalon 2779 / Speedy 4494 | Strata 71.9 / Avalon 103.0 / Speedy 104.3 | Strata 199.8 / Avalon 81.5 / Speedy 56.1 |
| Context | Writing Strata -> Speedy | Writing Avalon -> Speedy | Reading Strata -> Speedy | Reading Avalon -> Speedy | Wall Strata -> Speedy | Wall Avalon -> Speedy |
|---|---|---|---|---|---|---|
| 60K | +47% | +7% | 4.0x | 1.6x | 64% shorter | 26% shorter |
| 120K | +47% | +4% | 4.3x | 1.6x | 70% shorter | 29% shorter |
| 198K | +45% | +1% | 4.2x | 1.6x | 72% shorter | 31% shorter |
#Speedy#Avalon#Strata#qwen3.8 FN#NVIDIA GeForce RTX 4090#AMD Ryzen 9 7950X