ARC-Challenge
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Bits or permutations? Allocating a fixed inference budget
An arXiv paper compares lower-bit weights with averaging answer-option orders under a fixed inference budget on Qwen3-0.6B and Qwen3-1.7B. Using simulated round-to-nearest weight quantization and 300 items each from ARC-Easy, ARC-Challenge, MMLU and HellaSwag, the authors report that 6-bit weights scored 1.8 and 0.9 points above fp16 on the two models, although they do not explain the difference. At 4 bits, answer-slot skew increased, and averaging four cyclic orders recovered 12.6 points for the 0.6B model. The cost-matched analysis favors 6-bit weights plus up to four single-pass orderings when weights use at least 6 bits. All results are simulated and limited to two small models and the simplest quantizer.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my ARC-Challenge experienceAsk about ARC-Challenge
Nobody has shared their experience with ARC-Challenge yet.