MLE-Bench Lite

Headlines · 3

1. 5d
   MLE-World predicts ML experiment outcomes without running programs

   MLE-World, a world model fine-tuned from Qwen3.5-4B, predicts machine-learning programs’ terminal outputs—including error traces, training logs and validation scores—without executing them, alongside a confidence score. The paper reports that on 17 MLE-bench Lite tasks, its predictions matched real outputs for 77.9% of held-out programs, versus 41.1% for much larger prompted models. Five test-time-scaling policies guided by MLE-World produced up to 22.1% higher observed mean performance with less real execution than an agent without a world model.

   Hugging Face · Papers · 🔥 0
2. Oct 14
   MLExpert: a 35B mixture-of-experts agentic model for autonomous ML engineering

   An arXiv paper submitted on 15 October proposes MLExpert, a 35B mixture-of-experts agentic model trained for autonomous long-horizon machine learning engineering (MLE) optimization. The authors built the Kaggle-MLE Factory to generate executable tasks and an interaction harness, used hierarchical agentic supervised fine-tuning to transfer tool-use and coding skills, and applied Windowed Interaction RL (WIRL), which rewards the multi-turn decisions between successive experiment results as a shared execution window. The authors report that on MLE-Bench Lite MLExpert raises the base model's average medal rate from 37.9% to 72.7%, comparable to larger frontier models such as Kimi-K3 and GLM-5.3, with further gains reported on NatureBench Lite, agent scaffold optimization and algorithm optimization.

   Hugging Face · Papers · 🔥 0
3. Oct 13
   SwarmRSI evolves scientific agent swarms for recursive self-improvement

   An arXiv paper titled "SwarmRSI: Recursive Self-Improvement through Evolving Scientific Agent Swarm," submitted October 14 by Qianru Meng and 11 co-authors, introduces a framework that evolves scientific agent swarms for recursive self-improvement. It combines Feedback-Guided Swarm Mutation, which refines hypotheses and their implementations using experimental feedback from other agents, with Complementary Swarm Recombination, which reuses retained hypotheses and complementary program components across agents so partial results can support new research. Recursive Swarm Evolution ties both into a feedback loop guided by new experimental evidence. The authors report state-of-the-art performance on OpenProblems, NatureBench and MLE-Bench Lite, say accumulated research experience improves results on held-out tasks, and note that biologists reproduced its programs for follow-up biological analyses; these are self-reported results without cited independent replication.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my MLE-Bench Lite experienceAsk about MLE-Bench Lite