Realtime-Bench

Headlines · 2

1. 6d
   Realtime-Bench Tests 12 Frontier Models on Dynamic Tasks

   AIbase AI新闻 reports that NVIDIA and UT Austin introduced Realtime-Bench, a benchmark with 50 tasks across eight scenario types for assessing real-time perception, time-adaptive reasoning, and action timing. Researchers tested 12 frontier models under three environment-advancement clock protocols, and reportedly found that raising reasoning intensity did not reliably improve success. The report says GPT-6-Astra achieved a 44% task success rate when environments advanced according to actual reasoning time.

   AIbase AI新闻 · 🔥 0
2. Nov 2
   Realtime-Bench tests AI systems under real-time interaction

   Researchers introduced Realtime-Bench, a 50-task benchmark for evaluating general-purpose AI systems whose decisions and actions must respond to changing conditions in real time. It includes 30 web tasks and 20 game tasks, measuring temporal state awareness, adaptive deliberation, and time-aware action selection under wall-clock, paused-clock, and token-clock conditions. Import AI reported that GPT-6-Astra achieved the highest wall-clock pass rate at 36%, while high-reasoning models approached 70% to 80% with the clock paused.

   Import AI · 🔥 0

Experience and discussion from the community

Share my Realtime-Bench experienceAsk about Realtime-Bench