Artificial Analysis launches VIALS-AA visual-reasoning leaderboard
Artificial Analysis launched VIALS-AA, a leaderboard for Handshake AI’s VIALS benchmark, with 161 tasks across eight life-science and biotech domains. Models must interpret scientific artifacts—including gel blots, flow cytometry plots, plasmid maps, and phylogenies—and produce specific conclusions, with each task run three times in the Stirrup agent harness. Gemini 4 Carbon led with a 64% pass^3 score, followed by GPT-6 Astra at 63% and Claude Opus 5.5 at 61%, while Kimi K3.1 Pro led the open-weight models with 45%.
The leaderboard provides new evidence on frontier models’ visual reasoning, cost per task, open-weight performance, and safety refusals in life-science research, though the evaluation comes solely from Artificial Analysis.
We’re launching VIALS-AA, a leaderboard for Handshake AI’s VIALS benchmark for visual reasoning in life sciences and biotech research
Visual artifacts are central to how scientists analyze experiments and make research decisions, but interpreting them correctly remains a challenge for frontier models. VIALS’ 161 tasks span eight domains and need models to reason over artifacts including gel blots, flow cytometry plots, plasmid maps, and phylogenies. Each task requires models to interpret an artifact and draw a specific scientific conclusion.
We evaluate models by running each task three times in Stirrup, our open source agent harness, with code execution and image inspection tools. GPT-6 Luna (medium) grades each answer against the reference answer, and we report the share of tasks judged correct in all three repeats (pass^3).
Key takeaways:
➤ The top three models are separated by 3 points: Gemini 4 Carbon (high) from @GoogleDeepMind leads narrowly with 64%, followed by GPT-6 Astra (max) from @OpenAI at 63% and Claude Opus 5.5 (max with fallback) from @AnthropicAI at 61%
➤ Astra and Sol achieve similar scores to Opus and Sonnet at a much lower Cost per Task: Claude Opus 5.5 (max with fallback) averages $1.64 per task, ~29× as much as Sol (~$0.056). This price difference is significantly driven by turn use - GPT-6 Astra (max) and GPT-6.1 Sol (max) average ~8 turns per task, compared with ~27 for Claude Opus 5.5 (max with fallback) and ~25 for Claude Sonnet 5.5 (max with fallback)
#VIALS-AA#VIALS#Handshake AI#Artificial Analysis#Stirrup#Gemini 4 Carbon#GPT-6 Astra#GPT-6 Luna#GPT-6.1 Sol#Claude Opus 5.5#Claude Sonnet 5.5#Kimi K3.1 Pro