StreamEval benchmark tests 37 interactive streaming video generation models
The StreamEval Team posted an arXiv paper on 20 October 2026 introducing StreamEval, a unified benchmark for interactive streaming video generation that spans five tasks: avatar dialogue, avatar performance, world generation, instruction editing and reference editing. The benchmark contains 300 scripted test cases lasting 15 to 300 seconds, each specifying initial assets, incrementally arriving multimodal requests and explicit requirements for state changes and content preservation, and it scores models on task execution, state consistency, visual fidelity and audiovisual coordination through ten task-specific metrics on a 0-100 scale. The authors report evaluating 37 distinct models and finding that high visual quality and identity retention do not necessarily coincide with reliable request execution or sustained state consistency, with the highest task-specific scores reaching only 36.90 for Following among seven avatar-performance models, 25.93 for Following in text-conditioned world generation and 15.00 for Edit Success among three reported reference-editing models.
The authors conclude that reliable request execution under timing and preservation constraints is a central challenge for advancing omni-modal real-time interactive systems.
Computer Science > Computer Vision and Pattern Recognition
arXiv:2610.28402 (cs)
[Submitted on 20 Oct 2026]
Title:StreamEval: Evaluating Interactive Streaming Video Generation Models
Authors:StreamEval Team
View a PDF of the paper titled StreamEval: Evaluating Interactive Streaming Video Generation Models, by StreamEval Team