Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Developer claims near-5x token speedup for a 27B model on RTX 3090

AI summary

Wccftech reports that a developer on Reddit's LocalLLaMA forum said Qwen3.8-27B on an NVIDIA RTX 3090 went from 82 tok/s to 382 tok/s after releasing and using a custom inference engine, a gain of nearly 5x. On a 25,000-token document task, speed rose from 260 to 382 tokens per second, while ordinary chat reached about 133 tok/s, still above the original 82 tok/s. The post also describes extending the context window to 138,696 tokens by storing working memory in a more compact format; the optional longer checking step uses half the request slots and 8,000 tokens of context, and Wccftech says it has not independently verified the figures.

In-depth insight

The report says the bottleneck in token generation is fetching model weights from memory rather than computation, which Wccftech frames as potentially making older graphics chips more useful.

AIHardware

RTX 3090’s Token Generation Speed Gets Up To A 4.6x Boost Running A 27B LLM Through Just Inference Optimizations, Opening New Doors For Older GPUs

Omar Sohail•Oct 18, 2026 at 09:13am EDT

10 Comments

Image 3

Add to Preferres Sources

#NVIDIA#GeForce RTX 3090#Qwen3.8-27B#r/LocalLLaMA

Original · 5 min readRead the original
Group chat

No comments yet. Start the conversation.