Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Deepnote Benchmark Ranks AI Models for Proactive Workplace Help

AI summary

Deepnote’s Agent Initiative Benchmark tested 12 models in a simulated company Slack environment, asking them to determine when to respond, assist privately, or remain silent across 50 workplace tasks. GPT-6 Astra scored 8.97 and GPT-6.1 Sol scored 8.65, while Claude Fable 5.2 scored 8.17 at a per-task cost reported as 17 times Sol’s; the dataset and code were released openly. The results also showed that models often responded when they should have stayed silent, with unnecessary interventions consuming more time and money.

In-depth insight

Deepnote’s benchmark adds restraint, initiative, task quality, and operating cost to the evaluation of workplace agents, giving product teams a concrete basis for comparing models that continuously monitor collaboration tools.

ASI启示录 2026-11-14 17:06 北京

新智元报道 花430倍的钱,只多11%的分数!太夸张了。

新智元报道  

花430倍的钱,只多11%的分数!

太夸张了。数据协作平台Deepnote今天发布了一项评测结果——

#Deepnote#Agent Initiative Benchmark#OpenAI#Anthropic#GPT-6 Astra#GPT-6.1 Sol#Claude Fable 5.2#Claude Opus 5.5#GPT-6 Luna#DeepSeek V4.1 Flash#Mistral Large 4

Original · 6 min readRead the original
Group chat

No comments yet. Start the conversation.