创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Claude 修改权重改善 Silencio-2 控制

AI 总结

开发者 N1kuson 称,其使用 Claude Opus 5.5 诊断并直接修改 Silencio-2 的权重;该模型是基于 Kyutai Pocket TTS 的小型微调版本,支持行内情绪和声音标签。此次编辑未重新训练或执行梯度步骤,使留出测试集上的真实耳语比例从 5/24 提升至 22/24,情绪可分性从 0.45 提升至 0.71。该自述实验也记录了音频 glitches、耳语含混和指标局限。

深度见解

该实验展示了 LLM 智能体解释并直接编辑语音模型权重的一种实践路径,为缺少标注训练数据时的模型控制提供了替代思路。

I'm building Silencio-2, a small finetune of Kyutai's Pocket TTS. It adds inline tags for emotions (<happy>, <sad>, <angry>, <fear>, <surprised>, <whisper>) and for sounds like [laugh]. I simply didn't have enough tagged data to make them stable. <happy> worked, the others were hit or miss, and <whisper> gave an actual whisper in about 1 of 5 takes.

So instead of collecting more data, I tried the approach from this post on weight-space programming: an LLM agent inspects the model's weights, finds the mechanism behind the failure, edits a small number of weights directly, and keeps an edit only if it improves a held-out set. Here the agent was Claude (Opus 5.5). It wrote the probes, ran the experiments and built the edit. I steered and listened to the results.

What we found

  • The tag embeddings barely moved from the words they were initialized from: 60–80% cosine similarity to "sad", "whisper" and so on. Their norm doesn't matter either, because LayerNorm sits right before attention.
  • An attention probe showed that audio frames attend to <happy> 100–3000× more than to an average text token. For the other tags it's only 1–10×. The model seems to know what the tags mean, it just barely reads them.
  • Causal check: I boosted attention to <whisper> at inference time, directly in the activations (×30, all heads). That gave a whisper in 62% of takes with clean words. ×100 breaks speech. Boosting only the 3 "reader" heads did almost nothing.

#Silencio-2#Kyutai Pocket TTS#Claude Opus 5.5

原文约 3 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。