DecoyCount

Headlines · 1

1. 3d
   Elorian launches Glimpse and DecoyCount benchmark

   Elorian launched its first visual reasoning model, Glimpse, and co-released the DecoyCount benchmark with Mercor. The benchmark tests whether models genuinely count objects or rely on visual shortcuts and substitution heuristics. According to the accompanying paper, all evaluated frontier models scored below 50%, with GPT-6 Astra highest at 43.17%. Elorian CEO Andrew Dai said Glimpse scored 50.48% on the benchmark’s 1,660 items in an internal test that was not included in the paper. He said the evaluation images and text were withheld from Glimpse’s training and that a contamination check found no overlap in the data checked. Dai also said Glimpse’s result came from a single run without an agent or tool harness, while the six models in the paper were averaged across three runs. Elorian describes Glimpse as a visual thinking system that generates and edits images as part of its reasoning process, distinguishing the company’s approach from world models.

   Latent Space · 🔥 0

Experience and discussion from the community

Share my DecoyCount experienceAsk about DecoyCount