HOPE brings higher-order pruning to MoE language models
The authors introduce HOPE, a released expert-pruning method for Mixture-of-Experts language models that considers pairwise interactions rather than scoring each expert independently. They evaluated it on Qwen3.5-122B, Qwen3.5-35B, and GLM-4.5-Air, reporting the best average rank at 50% pruning and gains of up to 6.1% on agentic coding tasks over the next-best method. HOPE uses the same one-pass calibration as other pruning methods, while its added quadratic-programming step reportedly takes one to two seconds per layer.
The authors' results suggest that preserving cooperative groups of experts can improve aggressive pruning, where first-order approaches may leave performance on the table.
#HOPE#Qwen3.5-122B#Qwen3.5-35B#GLM-4.5-Air#REAP#SWE-Bench Verified#SWE-Bench Pro#Evol-CodeAlpaca#Tulu3