创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

作者报告 Hessian 精炼 Qwen3.8-27B GGUF 的 KL 散度降低

AI 总结

一名 Reddit 用户发布了一款经过 Hessian 精炼的 Qwen3.8-27B UD_4_XL GGUF,方法是应用 MACaQ 的前两项精炼技术,且无需修改推理软件。在单张 RTX 5090 的小型测试集上,作者报告其 KL 散度比 Unsloth 版本低 13.23%,推理速度相同;LiveCodeBench 高出 4%,但作者称这只是单个且不可靠的数据点。由于第三项 MACaQ 技术需要更新推理软件,因此未予采用。

深度见解

作者表示,使用兼容 GGUF 格式的用户只需替换模型文件,无需更新推理软件,也无需增加运行时计算量或显存占用,因此报告中的量化改进或可直接提供给本地模型用户。目前的结果尚未在该作者的小规模个人测试之外得到验证。

TLDR: Qwen3.8-27B UD_4_XL quant with 13.23% less KL divergence than unsloth (on my small testset, I just have a single 5090, feel free to test more). Technically it performs 4% better on my LiveCodeBench benchmark (but that would be a single data point, so it's not super-reliable). It is equally fast and using it in open-webui it "feels" better.

https://huggingface.co/christophedevriese/Qwen3.8-27B-Hessian-Refined-GGUF

I noticed from the MACaQ paper, which integrates changes proposed in Google/Deepmind papers, you can take the 3 components they use and just apply 1 and 2, and you just get >15% better KLD, 5% better performance on some benchmarks, while sacrificing absolutely nothing. Just switch the GGUF file. The numbers demonstrated here are the maximum that can be achieved in ~4 hours of optimizing on a single 5090, not the maximum that can be achieved. So it might be more than 15%, although I wouldn't necessarily hope for it.

This can be applied, without any patches to inference software, to text models, image models, even whisper ... the works. Switch GGUF file. Done/done.

The dream being that someone from unsloth looks at this and applies this to all their models (or just gives me access to a B300 machine for a day or two)

#Qwen3.8-27B Hessian-Refined GGUF#MACaQ#Qwen3.8-27B#Unsloth#Open WebUI#Google DeepMind

原文约 3 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。