AA-Robotics v0.1

Headlines · 2

1. 2d
   AA-Robotics v0.1: GPT-6 Astra tops robot-arm test, ~100x slower than humans

   Artificial Analysis released AA-Robotics v0.1 on October 14, a research-preview benchmark in which frontier models write code through coding agents to control a Hiwonder NexArm six-axis robot arm across eight tabletop tasks, including pressing a button, pouring a drink, writing and stacking blocks, observing the scene through four cameras and working zero-shot. First results: GPT-6 Astra 69.4, GPT-6.1 Sol 64.6, Gemini 4 Argon 55.3, Claude Opus 5.5 48.8 and Claude Fable 5.1 46.3, with partial credit meaning scores are not success rates. Models averaged roughly 30 minutes per task (Gemini 4 Argon about 52 minutes) against a human baseline of about 20 seconds, and cost ranged from $16.36 per task for Astra to $1.93 for Sol even though they differed by only 4.8 points. Grasping succeeded about 45% of the time for Astra and Fable 5.1 but 28% for Sol, and Sol built a passing block tower at around 17 minutes before continuing to adjust it and failing the task.

   机器之心 · 🔥 8
2. Oct 9
   Artificial Analysis teases AA-Robotics v0.1 benchmark for robot-arm control

   Artificial Analysis said on X that AA-Robotics v0.1 is coming soon, describing it as a benchmark for whether today's frontier language models can control a real robot arm zero-shot, with no coaching. The post gave no release date, model list, task design or scoring method, so it is a teaser rather than a published result. If it ships as described, the benchmark would extend language-model evaluation from text and software tasks into physical robot control.

   ArtificialAnlys (X) · 🔥 0

Experience and discussion from the community

Share my AA-Robotics v0.1 experienceAsk about AA-Robotics v0.1