Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Reddit user reports Speedy outperforming Avalon and Strata

AI summary

A Reddit user reported that the Speedy local-LLM inference engine outperformed Avalon on cold prefill, reading speed and total exchange time at 60K, 120K and 198K context lengths. In the same test on an RTX 4090 with 128 GB DDR5-3600 memory and a Ryzen 9 7950X, Speedy reportedly generated tokens about 45% faster than Strata and slightly faster than Avalon. The figures use ISTA IQ3_S quantization and were not independently verified.

In-depth insight

The post provides a concrete, configuration-specific benchmark signal for developers evaluating local LLM inference engines at long context lengths, though its results remain an unverified single-user report.

I had just finished setting up avalon, then i found speedy, which crushed avalon.

https://github.com/freriadoc/Avalon https://github.com/DevDock-AI/Speedy

Hardware: 4090, 128gb ddr5 3600, ryzen 9 7950x

ISTA IQ3_S quant:

ContextReading: cold prefill sReading: reported tok/sWriting: generation tok/sWhole exchange: wall s
60KStrata 54.7 / Avalon 22.4 / Speedy 13.7Strata — / Avalon 2678 / Speedy 4389Strata 75.0 / Avalon 102.5 / Speedy 110.0Strata 66.6 / Avalon 32.1 / Speedy 23.9
120KStrata 110.7 / Avalon 42.1 / Speedy 26.0Strata — / Avalon 2849 / Speedy 4622Strata 71.8 / Avalon 101.6 / Speedy 105.8Strata 123.2 / Avalon 52.0 / Speedy 37.0
198KStrata 187.2 / Avalon 71.2 / Speedy 44.1Strata — / Avalon 2779 / Speedy 4494Strata 71.9 / Avalon 103.0 / Speedy 104.3Strata 199.8 / Avalon 81.5 / Speedy 56.1
ContextWriting Strata -> SpeedyWriting Avalon -> SpeedyReading Strata -> SpeedyReading Avalon -> SpeedyWall Strata -> SpeedyWall Avalon -> Speedy
60K+47%+7%4.0x1.6x64% shorter26% shorter
120K+47%+4%4.3x1.6x70% shorter29% shorter
198K+45%+1%4.2x1.6x72% shorter31% shorter

#Speedy#Avalon#Strata#qwen3.8 FN#NVIDIA GeForce RTX 4090#AMD Ryzen 9 7950X

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.