More Reasoning Did Not Improve LLM Shopping Choices
An arXiv preprint benchmarks 10 frontier models on three product-choice benchmarks derived from WebShop, Amazon's SQID and Wayfair's WANDS. It reports that increasing reasoning effort did not improve decision quality in a statistically meaningful way, while stated confidence rose significantly. Most models were not well calibrated at every effort level. Errors were also strongly shared: when one model erred on a task, several others tended to err as well, and many selected the same incorrect product.
Computer Science > Artificial Intelligence
arXiv:2611.22033 (cs)
[Submitted on 17 Nov 2026]
Title:Thinking Harder Doesn't Lead to Better Choices: Reasoning Effort Increases the Confidence but Not the Correctness of LLM Shopping Agents
Authors:Adi Krishnamoorthy, X.Y. Han
View a PDF of the paper titled Thinking Harder Doesn't Lead to Better Choices: Reasoning Effort Increases the Confidence but Not the Correctness of LLM Shopping Agents, by Adi Krishnamoorthy and 1 other authors