Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Science Prediction Universe (SPU)

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 2

  1. SPU launches a wet-lab prediction benchmark for frontier AI models

    UniPat AI has introduced Science Prediction Universe (SPU), a benchmark with 116 tasks and 4,207 wet-lab experiments spanning chemistry, materials science, biology and medicine. According to QbitAI’s report, models must independently research experimental conditions and predict measured outcomes; in the first evaluation, Claude Opus 5.5 led with an average score of 0.486, while raising GPT-6 Astra’s inference effort left its prediction performance largely unchanged.

    量子位(原生 RSS) · 🔥 0
  2. UniPat AI leads launch of SPU wet-lab prediction benchmark

    According to a QbitAI report republished by 36Kr, UniPat AI led the launch of the Science Prediction Universe (SPU), a benchmark for evaluating AI systems that conduct autonomous research, use scientific tools and predict wet-lab outcomes. It contains 116 tasks and 4,207 experiments across chemistry, materials science, biology and medicine: 100 public tasks drawn from 85 recent papers and 16 private tasks based on unpublished collaborative experiments. The dataset and code have been open-sourced, while the public tasks are planned for release as complete Harbor task packages. Models receive eight hours per run, including an H20 GPU, and are scored by comparing predicted rankings with measured experimental rankings using Spearman correlation. The first evaluation covered 12 models in 13 configurations. Claude Opus 5.5 had the highest average score at 0.486 and exceeded the random threshold on about half of the tasks; the report said nearly all configurations exceeded that threshold on 34%–52% of tasks.

    36氪 人工智能 · 🔥 0

Experience and discussion from the community

Share my Science Prediction Universe (SPU) experienceAsk about Science Prediction Universe (SPU)

Nobody has shared their experience with Science Prediction Universe (SPU) yet.