创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

AMD Loom新增Radeon 9700支持,报告平均提速42%

AI 总结

一名Reddit发帖者称,实验性AMD Loom推理栈已新增Radeon 9700支持,并包含运行时、内核编译器和推理服务器。在48个可装入32GB显存的LLM测试中,其称Loom在预填充和解码指标上平均比llama.cpp快42%;此前在Strix Halo上的测试则平均快40%。发帖者还表示,所有受测Loom内核均由智能体技能生成。

深度见解

该报告让Radeon 9700成为这套实验性推理栈的新受测硬件,并提供了48个LLM的逐模型预填充与解码结果。

On Friday I posted about AMD Loom, a new experimental inference stack that includes a runtime, kernel compiler, and inference server. People can use the entire thing as a vertical stack, or the compiler/runtime can be integrated into any other inference project (I'm working on Gufo right now, the team is also working on a llamacpp backend).

I started with Strix Halo, where I ported the top-51 trending Q4_K_M LLMs from Hugging Face and got an average of 40% speedup over llama.cpp across decode and prefill metrics (picking the best results from ROCm or Vulkan backends).

I'm back today with Radeon 9700 results. I took the same 51 LLMs, cut it down to the 48 that fit in 32 GB of VRAM, and got an average of 42% speedup. Some of the data is posted to the table below, and all data is available to browse on https://loom-fun.github.io/loom-dev/index.html?gpu=radeon-r9700

All kernels were generated via agent skills:

部分来源内容未展示,可前往原文继续阅读。

#AMD Loom#AMD#Radeon 9700#llama.cpp#Strix Halo#ROCm#Gufo

原文约 2 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。