Developer releases Saber Rust inference engine for AMD R9700
AI summary
A developer released Saber, an LLM inference engine written in Rust and targeting AMD R9700 GPUs. It supports tensor parallelism settings of 1, 2 or 4, as well as Qwen3.8 27B. The developer reported 6,645 tokens per second for prefill and a 316 tokens per second decode peak on Qwen3.8 27B, using two R9700 GPUs, an AMD Ryzen 9900X and a Taichi X870E. The developer also claims Saber beats existing available engines on Qwen3.8 27B and Flash Next.
In-depth insight
Saber provides another AMD-targeted local LLM inference option, with the developer reporting high throughput and better performance than available engines.
| I know what you are thinking "another inference engine?", yeah there has been a flood of them I feel now. I have been running this for over a month now and I have been on the fence to even release it, but I decided to as I took a different approach that others currently available. Saber is a inference engine built in Rust targeting AMD R9700 GPU's. It currently supports TP 1, 2 or 4 and Qwen 3.8 27b and Flash next. Anyways here are the BetterBench results for Qwen3.8 27b, flash next is very fast also. Saber beats existing available engines on both 27b and flash next. BetterBench Qwen 3.8 27b Unsloth NVFP4 Results https://github.com/GGZ14/saber-engine https://x.com/bkuyper AMD Ryzen 9900X Taichi X870E 64GB RAM 2xR9700 submitted by /u/whodoneit1 [link] [comments] |