Community post reports 5,500 tok/s prompt processing on one RTX 5080
AI summary
A r/LocalLLaMA post reported that Basalt achieved 5,500 tokens/s in prompt processing while running IQ3_S Qwen3.8 Flash Next on a single RTX 5080. The author also cited peak generation rates of 275 tokens/s for code and tool calls and 190 tokens/s for reasoning traces, using 64GB DDR5-6400 memory and a Core Ultra 9 285K; no independent measurement was supplied.
In-depth insight
The report presents a detailed community benchmark for high-throughput local LLM inference on a single consumer GPU, although its figures remain self-reported and independently unverified.
| Code and tool calls can peak up to 275 tok/s - Reasoning traces up to 190 tok/s Hardware: 5080 16GB PCIe gen5x16, 64 DDR5 6400, Core Ultra 9 285K https://github.com/jesdga95/basalt Some PRs for the optimizations under my name are still pending (xyehya) but upstream already have solid gains & memory optimizations. and yes the IQ3_S is super usable. submitted by /u/yehyakar [link] [comments] |
#Basalt#Qwen3.8 Flash Next#NVIDIA GeForce RTX 5080#Intel Core Ultra 9 285K#OpenCode