社区测试称单卡RTX 5080达5500 token/s
AI 总结
r/LocalLLaMA一则帖子称，Basalt运行IQ3_S Qwen3.8 Flash Next时，在单张RTX 5080上实现每秒5500个token的提示处理速度。作者还报告代码与工具调用峰值每秒275个token、推理轨迹峰值每秒190个token，测试使用64GB DDR5-6400内存和Core Ultra 9 285K，但没有提供独立测量。
深度见解
该帖以较完整的信息展示了消费级GPU上的本地大模型高吞吐推理表现，但数据来自作者自报，尚无独立验证。
| Code and tool calls can peak up to 275 tok/s - Reasoning traces up to 190 tok/s Hardware: 5080 16GB PCIe gen5x16, 64 DDR5 6400, Core Ultra 9 285K https://github.com/jesdga95/basalt Some PRs for the optimizations under my name are still pending (xyehya) but upstream already have solid gains & memory optimizations. and yes the IQ3_S is super usable. submitted by /u/yehyakar [link] [comments] |
#Basalt#Qwen3.8 Flash Next#NVIDIA GeForce RTX 5080#Intel Core Ultra 9 285K#OpenCode