iPrefBench
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- iPrefBench tests how mobile agents infer preferences and ask users
The iPrefBench paper introduces an executable iOS benchmark for evaluating whether personalized mobile agents can distinguish information available from user history from information that must be requested. It contains 130 underspecified tasks across 26 apps and 3,051 activity events for one persona, with every omitted requirement labeled history-inferable or ask-only. Evaluations of seven frontier agents found action completion as high as 91.6% but full task success as low as 47.7%, with agents especially weak at requesting missing information.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my iPrefBench experienceAsk about iPrefBench
Nobody has shared their experience with iPrefBench yet.