AMD Loom新增Radeon 9700支持,报告平均提速42%
一名Reddit发帖者称,实验性AMD Loom推理栈已新增Radeon 9700支持,并包含运行时、内核编译器和推理服务器。在48个可装入32GB显存的LLM测试中,其称Loom在预填充和解码指标上平均比llama.cpp快42%;此前在Strix Halo上的测试则平均快40%。发帖者还表示,所有受测Loom内核均由智能体技能生成。
该报告让Radeon 9700成为这套实验性推理栈的新受测硬件,并提供了48个LLM的逐模型预填充与解码结果。
On Friday I posted about AMD Loom, a new experimental inference stack that includes a runtime, kernel compiler, and inference server. People can use the entire thing as a vertical stack, or the compiler/runtime can be integrated into any other inference project (I'm working on Gufo right now, the team is also working on a llamacpp backend).
I started with Strix Halo, where I ported the top-51 trending Q4_K_M LLMs from Hugging Face and got an average of 40% speedup over llama.cpp across decode and prefill metrics (picking the best results from ROCm or Vulkan backends).
I'm back today with Radeon 9700 results. I took the same 51 LLMs, cut it down to the 48 that fit in 32 GB of VRAM, and got an average of 42% speedup. Some of the data is posted to the table below, and all data is available to browse on https://loom-fun.github.io/loom-dev/index.html?gpu=radeon-r9700
All kernels were generated via agent skills:
- loom-skills repo: https://github.com/loom-fun/loom-skills
- deterministic SDK (runtime, compiler, server): https://github.com/ROCm/hrx-system
部分来源内容未展示,可前往原文继续阅读。