AutoSFT: a multi-agent system that automates SFT data research
A new arXiv paper introduces AutoSFT, a multi-agent system for automated data research in supervised fine-tuning. The authors report that a single agent given the full training stack converged on only one action — adjusting category proportions — while its context filled with execution output and was discarded mid-search. AutoSFT instead uses a meta agent that separates execution from analysis, gathers proposals from independent angles and calls a synthesis subagent for capabilities the data pool cannot cover. The authors report that the executor halves context loss, the analysis subagents lift the best IFEval by 6.5 points, and synthesis adds 3.4 IFEval and 2.9 IFBench points; on code tasks, analysis adds 3.7 MBPP+ over a single agent that does not improve. Every result comes from runs that retrain from the base model each round, with model, trainer and budget fixed within each comparison.
The authors conclude that automating SFT data research depends more on covering diverse data actions than on strengthening any single action.
Computer Science > Artificial Intelligence
arXiv:2610.22194 (cs)
[Submitted on 15 Oct 2026]
Title:AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning
Authors:Bingxiang He, Pengqi Sun, Shihao Peng, Shaoxuan Li, Huiyu Xiao, Hongjian Shi, Zixuan Fu, Ziqing Qiao, Yuxin Zuo, Cheng Qian, Xinqiang Yu, Jiayi Li, Xiusi Chen, Yizhen Song, Xiangyang Wang, Iofai Lei, Yudong Wang, Ning Ding, Zhiyuan Liu, Xiaozhi Wang, Chaojun Xiao
View a PDF of the paper titled AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning, by Bingxiang He and 20 other authors