hillclimb

Headlines · 1

1. 3d
   DeepRSI: Claude Fable 5.1 tops 48-hour AI research benchmark, Kimi K3 leads open weights

   Hillclimb, a Y Combinator-backed startup, released DeepRSI, a benchmark of 15 tasks drawn from real machine-learning research that tests models' recursive self-improvement. Models run offline with the same Pi Agent framework at maximum reasoning effort for 48 hours, proposing approaches, writing code and running experiments; scoring uses public plus private verifiers and is converted to Elo via a Bradley-Terry model. Claude Fable 5.1 ranked first with 1548 points, GPT-6 Astra second at 1232, and Moonshot's Kimi K3 third at 820 — the top open-weight model, ahead of Grok 4.7 and Gemini 3.8 Flash. Reported run costs were about $627 for Claude Fable 5.1, $404 for GPT-6 Astra and $159 for Kimi K3, though the article says these reflect test overhead rather than real research return on investment; hillclimb lists Jeff Dean among its investors.

   36氪 人工智能 · 🔥 0

Experience and discussion from the community

Share my hillclimb experienceAsk about hillclimb