Screwtape

Headlines · 1

1. 14h
   Redwood unveils Screwtape, a pipeline that edits agent transcripts to evade monitors

   Redwood Research introduced Screwtape, a pipeline that turns real agent transcripts into realistic attacks by inserting an attack and repeatedly refining it until an action-only monitor (one that does not read the chain of thought) no longer flags it; a separate LLM verifier, given the attack description and the transcript diff, judges whether the attack is realistic, harmful and plausibly successful. Running against monitors based on Opus 4.8 and Gemini 3.1 Pro over Redwood staff transcripts, 252 runs on 94 unique transcripts yielded 35 attacks that evaded their own monitors, six of which also evaded a held-out monitor from a frontier developer and were confirmed by human review. The authors caution that each iteration takes about 5–10 minutes, a run can take hours and most runs find no attack, that verifier mistakes mean human review is still needed, and that the pipeline produces no quantitative safety estimate, making it more useful for patching monitor blind spots than for evaluating monitors.

   Redwood Research · 🔥 7

Experience and discussion from the community

Share my Screwtape experienceAsk about Screwtape