METR challenges Anthropic's automated R&D risk assessment
METR reviewed Anthropic's August 2026 Risk Report using a private version of the document and other non-public data, agreeing that the July 15 models posed a low risk of direct catastrophic harm. METR nevertheless considered it plausible that the models had crossed Anthropic's early-warning threshold for a twofold acceleration in AI research progress, citing contestable judgment calls and understated uncertainty. METR also said Anthropic redacted most of the evidence it considered most informative, leaving the public report more dependent on METR's judgment.
The review challenges the transparency and evidentiary basis of a prominent frontier-AI safety assessment while maintaining that immediate catastrophic risk from the assessed models was low.
Redaction summary: As part of this assessment, Anthropic had rights to redact any non-public information in this post. Per our approach to transparency, we had the right to disclose if Anthropic exercised redaction rights, and whether or not any of our findings depended significantly on redacted information. All redactions that were important to our conclusions are described in the report below.
Executive summary
We agree with Anthropic that catastrophic risks from July 15th models were low, but we think it is plausible that they had crossed their early warning threshold at the time.
METR reviewed the “Risks from automated R&D” section of Anthropic’s public August 2026 Risk Report.1 We conducted this review as part of a high-access assessment of AI R&D acceleration, in which Anthropic shared a private version of the Risk Report along with extensive other non-public data. Alongside this review, we are publishing a holistic assessment of AI R&D acceleration at Anthropic. This was conducted by the same METR employees as this Risk Report Review.
Overall, we agree with Anthropic that by July 15th, internally deployed models posed low risk of directly automating R&D in key domains or causing acute catastrophic risks from acceleration of capabilities progress. However, the report’s claim that they had not crossed their early warning thresholds relies on contestable judgment calls that aren’t communicated clearly in the public report. While both METR’s researchers on this project and Anthropic’s relevant staff lean toward it not having been crossed, we think it is plausible that it had been, and we believe the public report overall understates the level of acceleration that was occurring.