Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Author reports lower KL divergence for Hessian-refined Qwen3.8-27B GGUF

AI summary

A Reddit user released a Hessian-refined Qwen3.8-27B UD_4_XL GGUF that applies the first two MACaQ refinement techniques without requiring inference-software patches. On a small test set using one RTX 5090, the author reports 13.23% lower KL divergence than the Unsloth version, equal inference speed, and a 4% LiveCodeBench improvement that the author describes as a single, unreliable data point. The third MACaQ technique was not used because it requires inference-software updates.

In-depth insight

The author says users of a compatible GGUF format can replace the model file without updating inference software or adding runtime compute or VRAM, potentially making the reported quantization improvements directly accessible to local-model users. The results remain unverified outside the author’s small personal test setup.

TLDR: Qwen3.8-27B UD_4_XL quant with 13.23% less KL divergence than unsloth (on my small testset, I just have a single 5090, feel free to test more). Technically it performs 4% better on my LiveCodeBench benchmark (but that would be a single data point, so it's not super-reliable). It is equally fast and using it in open-webui it "feels" better.

https://huggingface.co/christophedevriese/Qwen3.8-27B-Hessian-Refined-GGUF

I noticed from the MACaQ paper, which integrates changes proposed in Google/Deepmind papers, you can take the 3 components they use and just apply 1 and 2, and you just get >15% better KLD, 5% better performance on some benchmarks, while sacrificing absolutely nothing. Just switch the GGUF file. The numbers demonstrated here are the maximum that can be achieved in ~4 hours of optimizing on a single 5090, not the maximum that can be achieved. So it might be more than 15%, although I wouldn't necessarily hope for it.

This can be applied, without any patches to inference software, to text models, image models, even whisper ... the works. Switch GGUF file. Done/done.

The dream being that someone from unsloth looks at this and applies this to all their models (or just gives me access to a B300 machine for a day or two)

#Qwen3.8-27B Hessian-Refined GGUF#MACaQ#Qwen3.8-27B#Unsloth#Open WebUI#Google DeepMind

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.