创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Jev与Claude Opus混合路由降低身份解析成本

AI 总结

一项由厂商撰写的基准测试评估了 Jev 和 GPT-6 Luna 的身份解析评分能力;在作者所用环境中,两款模型单独使用的准确率均可接受,速度和成本也优于前沿模型。在更困难的测试中,系统把 Jev 的低置信度案例转交 Claude Opus 5.5 处理,每千个身份的成本为2.94美元、准确率达95.3%;Claude Opus 5.5 单独使用则为5.39美元和95.1%。作者将混合方案的优势归因于 Jev 更好的置信度校准,同时指出前沿模型单独使用时仍比决策模型更准确。

深度见解

该报告展示了以置信度为基础、将低成本决策模型与前沿模型结合的具体路由方案,但其成本、延迟和准确率结果依赖作者的身份解析测试环境。

We wanted to test whether LLMs and decision models, specifically Jev and Luna, could solve a hard problem that we encounter regularly (grading identity resolution results). We found that while both models performed okay out of the box (88.8–97.3% accuracy), implementing a hybrid model that combines Jev + Opus achieves frontier accuracy at > 4x lower costs.

Our other findings:

  • Luna performs better than Jev out of the box, but the gap disappears on harder cases.
  • Jev's confidence is better calibrated. Most of its mistakes come with low confidence its right. Most of Luna's mistakes come with high confidence. This enables Jev to perform better than Luna in a hybrid model harness.
  • Frontier models are still more accurate than decision models
  • Decision models are up to 90× cheaper and more than 10× faster out of the box. They cost $0.5 - $0.10 per 1,000 identities and answer in about 0.15 seconds, but trail Opus accuracy by about 6 percentage points on harder benchmarks.
Image 1

#Jev#GPT-6 Luna#Claude Opus 5.5#Gemini 3.1 Pro#GPT-6 Sol#OpenAI#Anthropic#Google#Hightouch#Decisions API

原文约 11 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。