Johns Hopkins
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Ai2 releases XYBench for testing assistants on mistaken assumptions
Ai2, with researchers from Carnegie Mellon, NVIDIA, and Johns Hopkins, released XYBench, a benchmark containing 8,115 questions about programming, cooking, home repair, and other practical tasks. It evaluates whether assistants identify misconceptions, address the user’s underlying goal, and prioritize effective solutions over the requested approach. Ai2 reports that Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B, and DeepSeek-V4 Pro often retained users’ mistaken approaches and that XYBench received the Best Resource Paper Award at EMNLP 2026.
Ai2 Blog · 🔥 0
Experience and discussion from the community
Share my Johns Hopkins experienceAsk about Johns Hopkins
Nobody has shared their experience with Johns Hopkins yet.