MLExpert: a 35B mixture-of-experts agentic model for autonomous ML engineering
An arXiv paper submitted on 15 October proposes MLExpert, a 35B mixture-of-experts agentic model trained for autonomous long-horizon machine learning engineering (MLE) optimization. The authors built the Kaggle-MLE Factory to generate executable tasks and an interaction harness, used hierarchical agentic supervised fine-tuning to transfer tool-use and coding skills, and applied Windowed Interaction RL (WIRL), which rewards the multi-turn decisions between successive experiment results as a shared execution window. The authors report that on MLE-Bench Lite MLExpert raises the base model's average medal rate from 37.9% to 72.7%, comparable to larger frontier models such as Kimi-K3 and GLM-5.3, with further gains reported on NatureBench Lite, agent scaffold optimization and algorithm optimization.
The work targets how to select useful supervision from long trajectories of costly experiments, and the authors say the learned optimization ability generalizes to other tasks, framing a concrete training route toward AI improving AI.
Computer Science > Artificial Intelligence
arXiv:2610.22344 (cs)
[Submitted on 15 Oct 2026]
Title:MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering
Authors:Shangheng Du, Jinxin Shi, Shuaiyu Zhang, Yiming Wu, Junpeng Zhao, Shiyang Feng, Xin Li, Liang He, Xiangchao Yan, Bo Zhang
View a PDF of the paper titled MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering, by Shangheng Du and 9 other authors
#MLExpert#Kaggle-MLE Factory#MLE-Bench Lite#NatureBench Lite#Kimi-K3#GLM-5.3#arXiv