19 Models Face Off in a Developer-Run Word-Association Game
AI summary
A Reddit developer reported logging 376 word-association games across 19 models, with each model taking both clue-giver and guesser roles against another model. GPT-6 Astra achieved a reported 91% good-guess rate and 1% bad-guess rate, while Gemini 3.8 Flash recorded 93% good guesses and no bad guesses; the developer said better guessing mattered more than clue grading, while more ambitious clues generally underperformed.
In-depth insight
The experiment offers a game-specific view of how frontier models balance clue generation and interpretation, although its self-reported methodology and limited setup restrict broader conclusions.
| I built a web version of a word-association party game (think Codenames; not affiliated) where any seat can be a human or an LLM. A clue-giver sees which of the 25 words are their team's and gives a one-word clue plus a number; a guesser on the same team picks words from it. One word is the assassin and loses the game instantly. Each model plays both roles on its own team, against a different model. I logged 376 games across 19 models. How the GPT models did, run with thinking at high effort: For comparison, at the same settings: Gemini 4 Argon 67 / 90% / <1%, Gemini 3.8 Flash 64 / 93% / 0%, Claude Opus 5.5 58 / 79% / 2%. So Gemini 3.8 Flash was the most accurate guesser, and the GPT models were close behind it, ahead of Claude. What surprised me across all 376 games: How a clue is graded: each board word gets an association score with the clue (1/3 Cohere embeddings, 2/3 two LLM raters that don't know which words are whose). A guessing model fitted on real games plays the turn out, and each revealed card is worth what it moved the win chance in saved games. The total is scaled to 0–100. "Bad guesses" hit the other team's word or the assassin. The ± is a 95% confidence interval. These newer models ran through my own subscriptions, not the paid API that players use. I'd like feedback on whether this is a fair way to grade a clue, and what you'd measure instead. You can play against these models at wordspies.com. Guests play a simple non-AI bot with no sign-up; a free account gets a first game with Claude Sonnet 5.5 on both teams, then 3 free AI games a day on smaller models. submitted by /u/Hardicecream17 [link] [comments] |
#WordSpies#GPT-6.1 Sol#GPT-6.1 Terra#GPT-6 Astra#Gemini 4 Argon#Gemini 3.8 Flash#Claude Opus 5.5#Claude Sonnet 5.5