Calibration-Free Top-k Selection Reduces MoE Expert Reads
A paper by Geunhyoung Kim and Sung Ju Hwang proposes top-k channel selection for Mixture-of-Experts decoding, avoiding the offline calibration data used by threshold-based selectors. The authors report that top-k matched or exceeded threshold variants in 40 of 43 comparisons at 13% and 18% read budgets, including all 23 comparisons on generation tasks, and avoided repetition-loop collapse observed with calibrated thresholds on Gemma 4. Their separate reader retained 98.1% of full-model accuracy while reading 17.8% of routed-expert weights across three post-trained MoE models. The authors also report that its Triton kernels ran MoE blocks 1.60–1.90 times faster than optimized vLLM at batch size one and up to 2.69 times faster at larger batch sizes.
MoE decoding is bottlenecked by loading routed-expert weights from memory during interactive inference at small batch sizes. The paper’s calibration-free selection and weight-based reader directly target that bottleneck, with author-reported reductions in expert-weight reads while retaining most measured accuracy.
Computer Science > Machine Learning
arXiv:2610.37386 (cs)
[Submitted on 27 Oct 2026]
Title:Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data
Authors:Geunhyoung Kim, Sung Ju Hwang
View a PDF of the paper titled Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data, by Geunhyoung Kim and 1 other authors
#Mixture-of-Experts#Triton#vLLM#Gemma 4#Geunhyoung Kim#Sung Ju Hwang