Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

RE-Bench

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 2

  1. Google DeepMind engineer: AI writes 75% of code, but self-improvement remains far off

    Google DeepMind engineer Haydn Belfield said more than 75% of Google's production code is written by AI, yet about 88% of code changes still require human oversight or review, with fully AI-delegated changes rising from 7.5% to 11.9%. In 1,389 internal Argon agent runs, unsupervised success fell sharply as task complexity rose; on RE-Bench, which contains 74 real machine-learning engineering tasks, Argon solved about half, versus 27% for Gemini 3.7 Flash. The team built a General Capability Index spanning 123 models and 92 benchmarks, finding Gemini capabilities improving about 0.1 GCI point per month with no statistically significant acceleration, and attributed Gemini 4 Argon's slightly above-trend result mainly to more compute and organizational investment rather than AI-driven acceleration. Separately, more than 20 researchers including Geoffrey Hinton, Yoshua Bengio and OpenAI's Jakub Pachocki warned in a paper that automating AI R&D could trigger an intelligence explosion compressing years of progress into months.

    36氪 人工智能 · 🔥 0
  2. Google DeepMind: no measurable acceleration from AI-driven AI research

    Google DeepMind engineer Haydn Belfield says there is no measurable acceleration of AI research driven by AI itself, and self-improving AI remains far off. Automation is already substantial: AI writes over 75 percent of production code, about 88 percent of code changes still need human oversight or review, and fully delegated changes rose from 7.5 to 11.9 percent. Across 1,389 internal Argon agent runs, success without human input dropped sharply as complexity grew, and on the in-house RE-Bench of 74 realistic ML engineering tasks, Argon solves about half versus 27 percent for Gemini 3.7 Flash. A General Capability Index covering 123 models and 92 benchmarks shows Gemini capabilities growing about 0.1 GCI points per month with no significant acceleration; Gemini 4 Argon sits slightly above the trend line, which the team attributes to compute and organizational investment. Belfield rates AI-driven research automation risk as low but is building measurement infrastructure, while over 20 researchers, including Geoffrey Hinton, Yoshua Bengio and OpenAI's Jakub Pachocki, warn of a possible intelligence explosion.

    The Decoder · 🔥 0

Experience and discussion from the community

Share my RE-Bench experienceAsk about RE-Bench

Nobody has shared their experience with RE-Bench yet.