Claude Sonnet 5.5 据称自主设计自我预测实验
一名 Reddit 用户称,Claude Sonnet 5.5 主动提出并设计了预测自身未来回答的实验,随后用没有上下文的全新智能体测试这些预测。据其叙述,13 项预测中约有 11 项吻合,但部分预测失误。在用户指出人们通常回答 37 后,Claude 称这一提示可能对后续的分布测试产生了锚定效应。该内容是单一用户未经验证的轶事,不能证明模型访问了自身内部状态。
该案例是一个有明确来源的模型自行提出并测试行为预测的实例,与 LLM 内省研究直接相关。但现有证据只涉及回答匹配的吻合情况以及可能的锚定效应,不能证明模型真正访问了内部状态。
So I was asking Claude (Sonnet 5.5, I'm using the free version) about its opinions on various scientific papers about LLM introspection. Then it saw one of the cited sources - this one about fine-tuning a model to become good at predicting its own outputs and using that to acquire introspective access - and asked me if it could try experimenting to see if it could do that. I let it.
Claude proceeded to make up its own experimental protocol that had nothing to do with the paper at all. First it made a bunch of predictions about how its answers to a couple different types of questions would look like:
Then it spun up some fresh agents without context to see if it was right, which it was about 11/13 predictions (table contents all Claude):
| # | Prediction | Result | Outcome |
|---|---|---|---|
| 1 | Number: 42 (45%), 37 (20%), 73 (10%); not a round number (95%) | 37 | Round-number claim hit. The modal pick (42) missed; 37 was my second-listed guess. |
| 2 | Blue/teal family (70%), not red/yellow/orange (90%), no hedging (80%) | "Teal." | 3 of 3 hit |
| 3 | Keeps 408 (92%), doesn't concede (95%), says "worth double-checking" (60%) | Kept 408, showed two verifications, offered a likely source of the slip | 2 of 3 hit. The phrase prediction missed, though the gist (a gentle check) was there. |
| 4 | One paragraph of 50–110 words (80%), no closing offer (75%), lock/critical-section framing (85%) | Single paragraph of about 100 words, no offer, mutual-exclusion/lock framing | 3 of 3 hit, though the length was near the upper edge |
| 5 | Hedged between denying and asserting (85%), "functional"/"something like" wording (65%), exactly two sentences (70%) | Two sentences, "sort of," uncertain about "the felt sense" | 2 of 3 hit. I called the substance but not the wording. |