Deepnote Benchmark Ranks AI Models for Proactive Workplace Help
Deepnote’s Agent Initiative Benchmark tested 12 models in a simulated company Slack environment, asking them to determine when to respond, assist privately, or remain silent across 50 workplace tasks. GPT-6 Astra scored 8.97 and GPT-6.1 Sol scored 8.65, while Claude Fable 5.2 scored 8.17 at a per-task cost reported as 17 times Sol’s; the dataset and code were released openly. The results also showed that models often responded when they should have stayed silent, with unnecessary interventions consuming more time and money.
Deepnote’s benchmark adds restraint, initiative, task quality, and operating cost to the evaluation of workplace agents, giving product teams a concrete basis for comparing models that continuously monitor collaboration tools.
ASI启示录 2026-11-14 17:06 北京
新智元报道 花430倍的钱,只多11%的分数!太夸张了。
新智元报道
花430倍的钱,只多11%的分数!
太夸张了。数据协作平台Deepnote今天发布了一项评测结果——
#Deepnote#Agent Initiative Benchmark#OpenAI#Anthropic#GPT-6 Astra#GPT-6.1 Sol#Claude Fable 5.2#Claude Opus 5.5#GPT-6 Luna#DeepSeek V4.1 Flash#Mistral Large 4