Tree-Ising用位级优化修复语言模型
AI 总结
Hadi Salloum和Ammar Ali在Tree-Ising论文中,将语言模型修复表述为对BF16权重的稀疏离散优化,并通过GPU模拟退火或模拟分岔求解独立的QUBO问题。作者报告称,修复Qwen3.5-0.8B两层后,MATH-500得分为54.6%,高于GPT-6 Astra智能体编辑的51.7%,且仅需四次前向请求和零个LLM词元。针对Qwen3.5-2B,作者还报告GSM8K准确率从21.6%升至69.4%,但目前没有独立复现。
深度见解
如果得到独立验证,这种方法可能以离散优化替代大额词元消耗和反复前向计算,为语言模型权重修复提供更低成本的路径。
Computer Science > Machine Learning
arXiv:2611.14210 (cs)
[Submitted on 10 Nov 2026]
Title:Tree-Ising: Repairing Language Models with Bit-Level Ising Edits
Authors:Hadi Salloum, Ammar Ali
View a PDF of the paper titled Tree-Ising: Repairing Language Models with Bit-Level Ising Edits, by Hadi Salloum and 1 other authors