AMD Loom Adds Radeon 9700 Support, Reports 42% Average Speedup
A Reddit poster reported that the experimental AMD Loom inference stack had added support for the Radeon 9700 and includes a runtime, kernel compiler, and inference server. In tests of 48 LLMs that fit within 32 GB of VRAM, the poster said Loom averaged 42% faster than llama.cpp across prefill and decode metrics. Earlier tests on Strix Halo reportedly produced an average 40% gain. The poster also said all tested Loom kernels were generated through agent skills.
The report adds a Radeon 9700 to the experimental stack's tested hardware and provides per-model prefill and decode results for 48 LLMs.
On Friday I posted about AMD Loom, a new experimental inference stack that includes a runtime, kernel compiler, and inference server. People can use the entire thing as a vertical stack, or the compiler/runtime can be integrated into any other inference project (I'm working on Gufo right now, the team is also working on a llamacpp backend).
I started with Strix Halo, where I ported the top-51 trending Q4_K_M LLMs from Hugging Face and got an average of 40% speedup over llama.cpp across decode and prefill metrics (picking the best results from ROCm or Vulkan backends).
I'm back today with Radeon 9700 results. I took the same 51 LLMs, cut it down to the 48 that fit in 32 GB of VRAM, and got an average of 42% speedup. Some of the data is posted to the table below, and all data is available to browse on https://loom-fun.github.io/loom-dev/index.html?gpu=radeon-r9700
All kernels were generated via agent skills:
- loom-skills repo: https://github.com/loom-fun/loom-skills
- deterministic SDK (runtime, compiler, server): https://github.com/ROCm/hrx-system
Some source content is omitted. Continue reading on the original site.