Singularity-Bench
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 2
- Claude Tops Singularity-Bench With Five Wins and One Draw
The Singularity-Bench preprint evaluates six AI-development tasks covering training, data selection, reinforcement-learning code, inference engines, agent runtimes, and GPU cluster scheduling. New智元 and AIbase report that Claude Opus 5.5 with Claude Code was the only combination above the human baseline, with five wins and one draw, while its generated inference engine reduced P90 first-token latency from 12.029 seconds with SGLang to 0.478 seconds on matched hardware. The researchers note that humans still defined each task, so the results do not yet demonstrate end-to-end recursive self-improvement.
新智元 · 🔥 3 - Opus 5.5 matches human baselines in CMU AI infrastructure benchmark
新智元 reports that a Carnegie Mellon University benchmark evaluated 11 agents on six end-to-end AI infrastructure tasks spanning pretraining, intermediate and post-training, inference, agent runtimes, and cluster orchestration. Opus 5.5 paired with Claude Code reached or exceeded the human baseline on all six tasks, with the reported systems matching or outperforming production tools such as SGLang and NVIDIA Dynamo on the same GPUs. The source also says Opus 5.5 scored 1.25 on the Singularity-Index, while GPT-6.1 Sol scored 0.969 and the other agents averaged 0.58.
虎嗅 AI · 🔥 3
Experience and discussion from the community
Share my Singularity-Bench experienceAsk about Singularity-Bench
Nobody has shared their experience with Singularity-Bench yet.