Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

MALT

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 1

  1. AI auditors detect agent misconduct, but high detection targets bring false alarms

    Andy Hall’s pilot study replicated and extended METR’s work by testing whether AI models can identify reward hacking and sandbagging in agent logs. At a 5% false-alarm rate, GPT-5 detected nearly 98% of reward-hacking cases, while the auditors caught 71% to 81% of sandbagging cases. The trade-off became more pronounced at demanding thresholds: in a second test using the R-Judge dataset, nearly all models stayed below 10% false alarms when targeting 95% detection, but the best model’s false-alarm rate rose to 15% at 99% detection. Hall concluded that AI auditors are better suited to estimating incident frequency than to reliably catching every incident, while noting that the analysis reflects his personal views. The reward-hacking results rest on only 47 cases, and most sandbagging cases were synthetic.

    The Decoder · 🔥 0

Experience and discussion from the community

Share my MALT experienceAsk about MALT

Nobody has shared their experience with MALT yet.