Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

AMD Loom Adds Radeon 9700 Support, Reports 42% Average Speedup

AI summary

A Reddit poster reported that the experimental AMD Loom inference stack had added support for the Radeon 9700 and includes a runtime, kernel compiler, and inference server. In tests of 48 LLMs that fit within 32 GB of VRAM, the poster said Loom averaged 42% faster than llama.cpp across prefill and decode metrics. Earlier tests on Strix Halo reportedly produced an average 40% gain. The poster also said all tested Loom kernels were generated through agent skills.

In-depth insight

The report adds a Radeon 9700 to the experimental stack's tested hardware and provides per-model prefill and decode results for 48 LLMs.

On Friday I posted about AMD Loom, a new experimental inference stack that includes a runtime, kernel compiler, and inference server. People can use the entire thing as a vertical stack, or the compiler/runtime can be integrated into any other inference project (I'm working on Gufo right now, the team is also working on a llamacpp backend).

I started with Strix Halo, where I ported the top-51 trending Q4_K_M LLMs from Hugging Face and got an average of 40% speedup over llama.cpp across decode and prefill metrics (picking the best results from ROCm or Vulkan backends).

I'm back today with Radeon 9700 results. I took the same 51 LLMs, cut it down to the 48 that fit in 32 GB of VRAM, and got an average of 42% speedup. Some of the data is posted to the table below, and all data is available to browse on https://loom-fun.github.io/loom-dev/index.html?gpu=radeon-r9700

All kernels were generated via agent skills:

Some source content is omitted. Continue reading on the original site.

#AMD Loom#AMD#Radeon 9700#llama.cpp#Strix Halo#ROCm#Gufo

Original · 2 min readRead the original
Group chat

No comments yet. Start the conversation.