Frontier models near ceiling on synthetic research test
A graphene and 2D materials researcher built a seven-part, 100-point synthetic materials-science test covering data cleaning, physics, manuscript critique, experimental design, and constrained writing. Across three fresh runs each, Gemini 4 Carbon, Gemini 4 Hydrogen, Claude Opus 5.5, and Claude Fable 5.2 produced mean scores from 98.5 to 100.0, with individual results spanning 97.8 to 100 and runs lasting 7–24 minutes. The r/MachineLearning post notes that the test was designed with Claude Opus 5.5's help and scored by one examiner using synthetic data. The author says the near-ceiling results are consistent with bounded tasks being saturated and plans a two-to-six-hour simulated-lab evaluation.
The experiment suggests frontier models can perform tightly bounded research-group workflows with near-perfect scores, while leaving open how to evaluate longer, open-ended agent work.