创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Visa购物基准:AI智能体测试运行失败率为42.8%

AI 总结

Visa称,其智能体商务基准让来自12家实验室的22个AI模型完成15项购物任务,涵盖电视、笔记本电脑和个人护理设备等商品选择。模型的测试运行有42.8%未通过,表现最佳的模型通过率为82.2%,最差为28.9%。每个模型对每个场景有三次尝试,三次尝试均通过的占比为34.2%。Visa没有公布受测模型名称;Fortune另行报道称,Google和Meta的智能体并列第一,Claude智能体排名第三,OpenAI智能体排名第六。所提供报道未独立核实Visa的测试结果。

深度见解

该基准考察购物智能体能否发现未提供的需求、正确决定购买或不购买、说明选择理由,并在重复场景中保持一致。Visa表示,结果显示一致性和可靠性仍是AI智能体用于商业活动时面临的问题。

Visa tested 22 AI models as shoppers, and they failed 42.8% of the time
Credit: Visa
Image Credits
Credit: Visa

Visa has launched an AI shopping benchmark, and in its first round the AI models failed 42.8% of their test runs. The Visa Agentic Commerce Benchmark put 22 AI models from 12 labs through 15 shopping tasks, the company said in a statement on Thursday.

The tasks included buying a TV, a laptop and a personal care appliance. The customer did not give every detail up front. Each AI had to work out what they needed by talking to them. If nothing on offer fitted, the AI had to decide not to buy. To rule out lucky guesses, it also had to find enough information to justify its choice.

#Visa#Visa Agentic Commerce Benchmark#Google#Meta#Claude#OpenAI

原文约 3 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。