Unsloth UD-IQ2_XXS
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Developer releases Rust and CUDA LLVQ implementation for Qwen3
Developer Pier-Jean Malandrino released an open-source Rust implementation of LLVQ with a custom CUDA kernel, along with quantized Qwen3 4B, 8B, and 14B files at roughly 2.7 bits per parameter. In the author's same-day L40S tests, LLVQ reportedly beat llama.cpp's IQ2_S on MMLU, GSM8K, and WikiText-2 quality at comparable bit rates, but decoded more slowly. The results remain self-reported, with limitations including calibration sensitivity, limited hardware coverage, higher long-prompt memory use, and support only for dense Qwen3 models up to 14B.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my Unsloth UD-IQ2_XXS experienceAsk about Unsloth UD-IQ2_XXS
Nobody has shared their experience with Unsloth UD-IQ2_XXS yet.