StreamEval
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 2
- StreamEval Benchmarks Real-Time Interactive Video Models
36Kr AI reports that Tsinghua University, NVIDIA, CMU, UC Berkeley, Princeton, NUS and other institutions jointly launched StreamEval, an open benchmark for real-time interactive video generation. It covers five task types, 300 test cases and 38 models, evaluating instruction execution, state consistency, visual fidelity and audio-visual coordination; the code, dataset and scoring rules are open for review and replication. The reported results show that strong video quality does not necessarily mean reliable continuous instruction following or long-term consistency.
36氪 人工智能 · 🔥 0 - StreamEval benchmark tests 37 interactive streaming video generation models
The StreamEval Team posted an arXiv paper on 20 October 2026 introducing StreamEval, a unified benchmark for interactive streaming video generation that spans five tasks: avatar dialogue, avatar performance, world generation, instruction editing and reference editing. The benchmark contains 300 scripted test cases lasting 15 to 300 seconds, each specifying initial assets, incrementally arriving multimodal requests and explicit requirements for state changes and content preservation, and it scores models on task execution, state consistency, visual fidelity and audiovisual coordination through ten task-specific metrics on a 0-100 scale. The authors report evaluating 37 distinct models and finding that high visual quality and identity retention do not necessarily coincide with reliable request execution or sustained state consistency, with the highest task-specific scores reaching only 36.90 for Following among seven avatar-performance models, 25.93 for Following in text-conditioned world generation and 15.00 for Edit Success among three reported reference-editing models.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my StreamEval experienceAsk about StreamEval
Nobody has shared their experience with StreamEval yet.