Science Prediction Universe (SPU)
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 2
- SPU launches a wet-lab prediction benchmark for frontier AI models
UniPat AI has introduced Science Prediction Universe (SPU), a benchmark with 116 tasks and 4,207 wet-lab experiments spanning chemistry, materials science, biology and medicine. According to QbitAI’s report, models must independently research experimental conditions and predict measured outcomes; in the first evaluation, Claude Opus 5.5 led with an average score of 0.486, while raising GPT-6 Astra’s inference effort left its prediction performance largely unchanged.
量子位(原生 RSS) · 🔥 0 - UniPat AI leads launch of SPU wet-lab prediction benchmark
According to a QbitAI report republished by 36Kr, UniPat AI led the launch of the Science Prediction Universe (SPU), a benchmark for evaluating AI systems that conduct autonomous research, use scientific tools and predict wet-lab outcomes. It contains 116 tasks and 4,207 experiments across chemistry, materials science, biology and medicine: 100 public tasks drawn from 85 recent papers and 16 private tasks based on unpublished collaborative experiments. The dataset and code have been open-sourced, while the public tasks are planned for release as complete Harbor task packages. Models receive eight hours per run, including an H20 GPU, and are scored by comparing predicted rankings with measured experimental rankings using Spearman correlation. The first evaluation covered 12 models in 13 configurations. Claude Opus 5.5 had the highest average score at 0.486 and exceeded the random threshold on about half of the tasks; the report said nearly all configurations exceeded that threshold on 34%–52% of tasks.
36氪 人工智能 · 🔥 0
Experience and discussion from the community
Share my Science Prediction Universe (SPU) experienceAsk about Science Prediction Universe (SPU)
Nobody has shared their experience with Science Prediction Universe (SPU) yet.