创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Gemini Decision 1的800个场景评测

AI 总结

一名作者对谷歌Gemini Decision 1的评测覆盖了24个类别和800个会话。该模型不生成文本,而是返回布尔值、分数或类别;在狭窄检查中表现较强,提示注入检测准确率达到99.7%,判断哪些内容值得记忆的准确率为88%。但它在宽泛、主观的判断上可靠性较低:预测用户反对反应的准确率仅为46%,而在切分代码文件以供检索时,确定性代码表现更好。作者测得单题约需180毫秒,一次处理100个问题约需250毫秒。测试使用了作者自己的会话和一个公开数据集,并非穷尽性评测。

深度见解

报告结果支持将决策模型用作狭窄分类或评分检查的快速、低成本过滤层,同时将宽泛的主观判断留给完整语言模型,并在确定性代码效果更好时优先使用代码。由于评测仅覆盖一名作者的会话和一个公开数据集,这些结果并非穷尽性基准。

With all the excitement around large language models over the last few years, it's easy to forget that machine learning has been around a lot longer than chat models, and that other models are built for very different jobs.

Today, most AI systems leave a lot of efficiency on the table by calling a full LLM just to get a simple true or false response or a basic classification.

Google recently released Gemini Decision 1, also known as a 'System 1' model. Instead of generating text token by token like a chat model, it reads your input in one pass and returns one of three structured statistical responses:

  • True or false
  • A score
  • A category (classification)

Why decision models are fast and cheap

Because there are no generated text tokens, both the speed and the pricing look very different from a standard LLM.

#Gemini Decision 1#Gemini Decisions API#Google

原文约 4 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。