GLM-4.5-Air
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 3
- tandem.cpp adds AMD ROCm support and large-model CPU/GPU offloading
The developer of tandem.cpp announced that its llama.cpp patch set can now build with HIP for AMD ROCm, while keeping frequently used MoE experts on the GPU and offloading remaining work to the CPU. In developer-run tests, DeepSeek-V4.1-Flash IQ2_XS with 139 GiB of experts reached 14.2 decode tokens per second on a system with an RTX 5070 Ti and 64 GB RAM, compared with 7.4 for llama.cpp. The AMD RX 7900 XTX test also included Qwen3.8 IQ3_S with MTP, but the reported performance figures have not been independently verified and Vulkan support is not yet available.
r/LocalLLaMA · 🔥 0 - HOPE brings higher-order pruning to MoE language models
The authors introduce HOPE, a released expert-pruning method for Mixture-of-Experts language models that considers pairwise interactions rather than scoring each expert independently. They evaluated it on Qwen3.5-122B, Qwen3.5-35B, and GLM-4.5-Air, reporting the best average rank at 50% pruning and gains of up to 6.1% on agentic coding tasks over the next-best method. HOPE uses the same one-pass calibration as other pruning methods, while its added quadratic-programming step reportedly takes one to two seconds per layer.
Hugging Face · 🔥 0 - tandem.cpp accelerates CPU-assisted MoE decoding on consumer GPUs
The independent tandem.cpp project has released patches for llama.cpp that keep frequently used MoE experts on the GPU while the CPU handles the rest, targeting single-user decoding when a model does not fit in VRAM. The project reports that Qwen3.8-Flash-Next reached 97.3 decoding tokens per second with greedy sampling on an i5-12400F system with 64 GB DDR4 memory and an RTX 4090, compared with 29.8 tokens per second for llama.cpp. It also reports gains for gpt-oss-120b and other MoE models, although the implementation is tuned specifically for Qwen3.8.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my GLM-4.5-Air experienceAsk about GLM-4.5-Air
Nobody has shared their experience with GLM-4.5-Air yet.