创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Artificial Analysis推出VIALS-AA视觉推理榜单

AI 总结

Artificial Analysis推出VIALS-AA榜单,用于评测Handshake AI的VIALS基准;该基准包含生命科学与生物技术八个领域的161项任务。模型需要解读凝胶印迹、流式细胞术图、质粒图谱和系统发育树等科学图像,并在Stirrup智能体框架中重复运行三次;最终Gemini 4 Carbon以64%的pass^3成绩居首,GPT-6 Astra和Claude Opus 5.5分别为63%和61%。

深度见解

该榜单为生命科学研究中的模型视觉推理、单任务成本、开放权重表现和安全拒答提供了新数据,但评测结果仅来自Artificial Analysis一方。

We’re launching VIALS-AA, a leaderboard for Handshake AI’s VIALS benchmark for visual reasoning in life sciences and biotech research

Visual artifacts are central to how scientists analyze experiments and make research decisions, but interpreting them correctly remains a challenge for frontier models. VIALS’ 161 tasks span eight domains and need models to reason over artifacts including gel blots, flow cytometry plots, plasmid maps, and phylogenies. Each task requires models to interpret an artifact and draw a specific scientific conclusion.

We evaluate models by running each task three times in Stirrup, our open source agent harness, with code execution and image inspection tools. GPT-6 Luna (medium) grades each answer against the reference answer, and we report the share of tasks judged correct in all three repeats (pass^3).

Key takeaways:

➤ The top three models are separated by 3 points: Gemini 4 Carbon (high) from @GoogleDeepMind leads narrowly with 64%, followed by GPT-6 Astra (max) from @OpenAI at 63% and Claude Opus 5.5 (max with fallback) from @AnthropicAI at 61%

➤ Astra and Sol achieve similar scores to Opus and Sonnet at a much lower Cost per Task: Claude Opus 5.5 (max with fallback) averages $1.64 per task, ~29× as much as Sol (~$0.056). This price difference is significantly driven by turn use - GPT-6 Astra (max) and GPT-6.1 Sol (max) average ~8 turns per task, compared with ~27 for Claude Opus 5.5 (max with fallback) and ~25 for Claude Sonnet 5.5 (max with fallback)

#VIALS-AA#VIALS#Handshake AI#Artificial Analysis#Stirrup#Gemini 4 Carbon#GPT-6 Astra#GPT-6 Luna#GPT-6.1 Sol#Claude Opus 5.5#Claude Sonnet 5.5#Kimi K3.1 Pro

原文约 2 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。