Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

LifeVal tests personalized AI assistants on everyday tasks

AI summary

OpenAI introduced LifeVal, a benchmark for evaluating personalized AI assistants on everyday consumer tasks involving browsing and computer use. The reports conflict on whether it contains 148 or 169 tasks. One report says that, among conditions requiring information from prior ChatGPT conversations, the average success rate was 33.33%. It also says that moving background information into conversation history reduced the relevant rate from 55.8% to 41.5%. The paper reportedly does not identify which models were used for the personalization analysis. LifeVal’s paper and results were released, but its full task dataset and evaluation code were not.

In-depth insight

The results show that access to personal information does not necessarily mean an AI assistant will find and use the relevant details in prior conversations, a central requirement for completing personalized tasks.

移至主內容

文/李建興|2026-11-05 發表

OpenAI公布**日常生活任務基準測試LifeVal**,研究發現,即使AI助理能存取使用者過往的ChatGPT對話,仍可能漏掉完成任務所需的個人資訊。在被判定需要從歷史對話取得資訊的評分條件中,平均達標率僅33.33%,反映出運用過往對話的能力仍有不足。

LifeVal收錄169項任務,涵蓋照護、教育、購物、個人財務與政府服務。研究團隊邀請具備相關經驗的參與者設計任務,結合經去識別化處理的ChatGPT對話與記憶、參考檔案及額外背景資訊,測試AI透過瀏覽器與電腦操作完成工作。評測主要在模擬環境中進行,包括尋找符合家庭條件的幼兒園、安排防洪志工及搜尋住宅等情境。

每項LifeVal任務都設有多個評分條件,檢查AI是否完成指定操作,以及結果是否符合使用者需求。研究團隊分析模型運用個人資訊的能力,發現被判定需要使用ChatGPT記憶的評分條件,平均達標率為47.8%,需要使用額外背景資訊的達標率則為55.8%。

#OpenAI#LifeVal#ChatGPT#GPT-6 Astra#GPT-5.6#GPT-5.2

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.