MLE-World predicts ML experiment outcomes without running programs
MLE-World, a world model fine-tuned from Qwen3.5-4B, predicts machine-learning programs’ terminal outputs—including error traces, training logs and validation scores—without executing them, alongside a confidence score. The paper reports that on 17 MLE-bench Lite tasks, its predictions matched real outputs for 77.9% of held-out programs, versus 41.1% for much larger prompted models. Five test-time-scaling policies guided by MLE-World produced up to 22.1% higher observed mean performance with less real execution than an agent without a world model.
The paper demonstrates that a learned predictor can estimate ML program outcomes with one inference call instead of a training run, providing a concrete approach to scaling agent exploration while limiting real execution.
Computer Science > Machine Learning
arXiv:2610.37175 (cs)
[Submitted on 27 Oct 2026]
Title:MLE-World: Predicting Machine Learning Experiment Outcomes with a Learned World Model
Authors:Brian Ko, Han Wang, Hejia Geng, Li Erran Li, Sherry Yang
View a PDF of the paper titled MLE-World: Predicting Machine Learning Experiment Outcomes with a Learned World Model, by Brian Ko and 4 other authors