开发者发布面向 AMD R9700 的 Saber Rust 推理引擎
AI 总结
一名开发者发布了用 Rust 编写、面向 AMD R9700 GPU 的 LLM 推理引擎 Saber。该引擎支持 1、2 或 4 路张量并行,以及 Qwen3.8 27B。开发者报告,在两块 R9700 GPU、AMD Ryzen 9900X 和 Taichi X870E 组成的测试环境中,Qwen3.8 27B 的预填充速度达到每秒 6,645 个 token,解码峰值达到每秒 316 个 token。开发者还称,Saber 在 Qwen3.8 27B 和 Flash Next 上的表现优于现有可用引擎。
深度见解
Saber 为 AMD 平台提供了另一种本地 LLM 推理选择;开发者报告了较高吞吐量,并称其表现优于现有可用引擎。
| I know what you are thinking "another inference engine?", yeah there has been a flood of them I feel now. I have been running this for over a month now and I have been on the fence to even release it, but I decided to as I took a different approach that others currently available. Saber is a inference engine built in Rust targeting AMD R9700 GPU's. It currently supports TP 1, 2 or 4 and Qwen 3.8 27b and Flash next. Anyways here are the BetterBench results for Qwen3.8 27b, flash next is very fast also. Saber beats existing available engines on both 27b and flash next. BetterBench Qwen 3.8 27b Unsloth NVFP4 Results https://github.com/GGZ14/saber-engine https://x.com/bkuyper AMD Ryzen 9900X Taichi X870E 64GB RAM 2xR9700 submitted by /u/whodoneit1 [link] [comments] |