GeForce RTX 3090

Headlines · 1

1. Oct 18
   Developer claims near-5x token speedup for a 27B model on RTX 3090

   Wccftech reports that a developer on Reddit's LocalLLaMA forum said Qwen3.8-27B on an NVIDIA RTX 3090 went from 82 tok/s to 382 tok/s after releasing and using a custom inference engine, a gain of nearly 5x. On a 25,000-token document task, speed rose from 260 to 382 tokens per second, while ordinary chat reached about 133 tok/s, still above the original 82 tok/s. The post also describes extending the context window to 138,696 tokens by storing working memory in a more compact format; the optional longer checking step uses half the request slots and 8,000 tokens of context, and Wccftech says it has not independently verified the figures.

   Wccftech · 🔥 0

Experience and discussion from the community

Share my GeForce RTX 3090 experienceAsk about GeForce RTX 3090