OpenAI推出LifeVal日常任务基准
AI 总结
OpenAI推出LifeVal,用于评估AI能否完成照护、健康、教育、家庭管理、个人财务、购物和获取政府服务等实用个人助理工作流。OpenAI表示,该基准参考时间使用调查并基于3,995名受访者构建,过去一年已用于追踪模型改进;公开示例包括幼儿园筛选93.1%、组织防洪志愿者98.0%和制定糖尿病家人膳食计划90.4%。不过,LifeVal目前主要仍是内部评估工具,完整任务集和评分代码尚未公开。
深度见解
LifeVal将评测重点转向可执行、可交付的日常任务,但因完整基准和评分代码未公开,其示例成绩暂无法独立复现。
For AGI to benefit all of humanity, it should help people with the everyday tasks that matter most to them. So we built LifeVal, a new benchmark shaped by feedback from thousands of ChatGPT users on where they most want help from a personal assistant—from finding childcare to accessing government services to managing personal finances.
Over the past year, we’ve used LifeVal to measure and improve how well our models handle these everyday tasks.
https://openai.com/index/lifeval/