Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Meta's Muse Wonder tops coding benchmarks but flunks a lava lamp test

AI summary

Meta released Muse Wonder and Muse Wonder Full on Oct 12, and official figures put them in the top tier on several benchmarks: Artificial Analysis Intelligence Index scores of 52 and 56, between Claude Opus 5.5 (58) and GPT-6 Astra (53), plus 57.5 on SWE-Marathon for the Full version versus 46.3 for Claude Opus 5.5. But in an open-ended real-world test by evaluator BridgeMind, the model failed to build a lava lamp with a transparent glass shell, moving lava and realistic lighting, producing what the article calls an ordinary-looking desk lamp; BridgeMind accused it of "benchmaxxing," and Elon Musk replied "Hmm" on X. The article also notes that SWE-Marathon actually grades a full coding-agent setup, including Codex and Claude Code, rather than a bare model, and that Meta's focus has shifted from the Llama open ecosystem toward a personal assistant built around long-term memory and sustained task execution.

In-depth insight

If high-scoring models keep stumbling on open-ended real-world tasks, both user trust in AI capability and the credibility of benchmarks come under question.

MetaMuse WonderMuse Wonder Full

Original · 8 min readRead the original
Group chat

No comments yet. Start the conversation.