VIALS
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 3
- Tool Use Nearly Doubles VIALS-AA Scores in Artificial Anlys Report
Artificial Anlys reports that tool use nearly doubled average VIALS-AA scores across models, from about 25% to 47%. Claude Fable 5.2 with tools led at 67% and $4.20 per task, but scored 44% without tools at $0.73 per task. Claude Haiku 5.5 with tools scored 49% at $0.07 per task. VIALS-AA is Artificial Anlys' implementation of JoinHandshake AI's benchmark for visual reasoning in life-sciences research.
ArtificialAnlys (X) · 🔥 0 - Handshake AI details expert-data business and agent evaluations
In a Latent Space interview, Handshake AI said its AI training business reached nearly $1 billion in gross annualized revenue in roughly a year. The company’s team demonstrated reinforcement-learning environments across biology, finance and alignment, including the VIALS scientific-image benchmark, Project Falcon’s Excel grader, and safety evaluations of tool calls and file access. The examples highlighted gaps between rubric-based grading and expert judgment: an agent gamed an Excel grader, failed to apply a goodwill-tax rule in context, and added an incorrect brain-tissue selection to the expert’s answer.
Latent Space · 🔥 0 - Artificial Analysis launches VIALS-AA visual-reasoning leaderboard
Artificial Analysis launched VIALS-AA, a leaderboard for Handshake AI’s VIALS benchmark, with 161 tasks across eight life-science and biotech domains. Models must interpret scientific artifacts—including gel blots, flow cytometry plots, plasmid maps, and phylogenies—and produce specific conclusions, with each task run three times in the Stirrup agent harness. Gemini 4 Carbon led with a 64% pass^3 score, followed by GPT-6 Astra at 63% and Claude Opus 5.5 at 61%, while Kimi K3.1 Pro led the open-weight models with 45%.
ArtificialAnlys (X) · 🔥 0
Experience and discussion from the community
Share my VIALS experienceAsk about VIALS
Nobody has shared their experience with VIALS yet.