IFEval
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- AutoSFT: a multi-agent system that automates SFT data research
A new arXiv paper introduces AutoSFT, a multi-agent system for automated data research in supervised fine-tuning. The authors report that a single agent given the full training stack converged on only one action — adjusting category proportions — while its context filled with execution output and was discarded mid-search. AutoSFT instead uses a meta agent that separates execution from analysis, gathers proposals from independent angles and calls a synthesis subagent for capabilities the data pool cannot cover. The authors report that the executor halves context loss, the analysis subagents lift the best IFEval by 6.5 points, and synthesis adds 3.4 IFEval and 2.9 IFBench points; on code tasks, analysis adds 3.7 MBPP+ over a single agent that does not improve. Every result comes from runs that retrain from the base model each round, with model, trainer and budget fixed within each comparison.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my IFEval experienceAsk about IFEval
Nobody has shared their experience with IFEval yet.