GPT-6.1 Sol在个人Salesforce代理测试中落后
AI 总结
一名Salesforce开发者让GPT-6.1 Sol、Claude Opus、Claude Sonnet、Claude Haiku和DeepSeek完成50项代理任务,并按数据库的实际变更而非模型回复评分。GPT-6.1 Sol得分72%,落后于Opus的87%、Sonnet的85%、DeepSeek的82%和Haiku的80%。它发现了阶段变更后的所有概率重置,并在长流程多步骤任务中最善于报告系统自动执行的变更,36项中成功21项。测试者称,差距主要来自未能恢复交易的原始关闭日期,或重开交易时未说明日期已过去六个月等任务。
| I've been working on a project testing AI agents in a Salesforce dev org and wanted to share how GPT-6.1 Sol did, since the results were mixed. Five models (GPT-6.1 Sol, Claude Opus, Sonnet, Haiku and DeepSeek) worked through 50 tasks, and each attempt was graded on what actually changed in the database, not on the model's reply. Some of the findings for GPT-6.1 Sol: If anyone's interested: Write-up: https://outsideengineer.substack.com/p/evaluating-ai-agents-on-salesforce Code and transcripts: https://github.com/brianna-rose05/crm-agent-bench submitted by /u/TruthDeep8659 [link] [comments] |
#GPT-6.1 Sol#Claude Opus#Claude Sonnet#Claude Haiku#DeepSeek#Salesforce