iPrefBench tests how mobile agents infer preferences and ask users
The iPrefBench paper introduces an executable iOS benchmark for evaluating whether personalized mobile agents can distinguish information available from user history from information that must be requested. It contains 130 underspecified tasks across 26 apps and 3,051 activity events for one persona, with every omitted requirement labeled history-inferable or ask-only. Evaluations of seven frontier agents found action completion as high as 91.6% but full task success as low as 47.7%, with agents especially weak at requesting missing information.
The benchmark tests a distinct capability behind mobile automation: completing the right action for the right user while avoiding unsupported assumptions and unnecessary questions.
Computer Science > Computation and Language
arXiv:2610.34735 (cs)
[Submitted on 26 Oct 2026]
Title:iPrefBench: A User-Centric Benchmark for Personalized Mobile Use Agents
Authors:Yuting Ning, Ruijia Cheng, Sushobhan Ghosh, Peitong Duan, HungShi Lin, Abhishek Sundararajan, Shreyash Pandey, Huan Sun, Jeffrey Nichols, Eldon Schoop
View a PDF of the paper titled iPrefBench: A User-Centric Benchmark for Personalized Mobile Use Agents, by Yuting Ning and 9 other authors