Nanbeige4.3-33B GGUF Quantizations Span 35.2GB to 13.7GB
AI summary
A r/LocalLLaMA post presents quantized GGUF builds of Nanbeige4.3-33B ranging from approximately 35.2GB to 13.7GB. It identifies the 20.3GB AD-IQ4_XS build as the preferred option for 32GB hardware and reports a 92.0% top-1 result relative to BF16, though the benchmark methodology and result are not independently verified.
In-depth insight
The reported quantization ladder provides concrete file-size options for local deployment, including a 20.3GB AD-IQ4_XS build identified for 32GB hardware.
| Quant ladder of Nanbeige's new 33B model from 35.2GB down to 13.7GB For 32GB hardware the sweet spot is 20.3GB AD-IQ4_XS with a top-1% of 92.0% compared to BF16 submitted by /u/gladkos [link] [comments] |