llama.cpp PR fuses MoE matrix multiplications and GLU in MMQ
A llama.cpp pull request proposes fusing MoE gate/up matrix multiplications and GLU in the CUDA MMQ path. The PR reports prompt-processing gains of 5.7%–17.2% across DGX Spark and RTX 5090 systems using several Qwen MoE model and quantization configurations. The supplied report does not establish that the change has been merged or released, and the benchmark figures have not been independently verified.
The PR combines specific MoE operations within the CUDA MMQ path and reports prompt-processing improvements across multiple hardware, model, and quantization configurations, making it a potentially relevant optimization for llama.cpp inference users. Its actual impact remains unverified.
| download your CUDA/MoE prompt-processing speedup today: submitted by /u/jacek2023 [link] [comments] |
#llama.cpp#CUDA#NVIDIA DGX Spark#NVIDIA GeForce RTX 5090#Qwen3-30B-A3B#Qwen3.6-35B-A3B