StreamEval Benchmarks Real-Time Interactive Video Models
36Kr AI reports that Tsinghua University, NVIDIA, CMU, UC Berkeley, Princeton, NUS and other institutions jointly launched StreamEval, an open benchmark for real-time interactive video generation. It covers five task types, 300 test cases and 38 models, evaluating instruction execution, state consistency, visual fidelity and audio-visual coordination; the code, dataset and scoring rules are open for review and replication. The reported results show that strong video quality does not necessarily mean reliable continuous instruction following or long-term consistency.
StreamEval provides a public, unified evaluation framework for measuring interaction and state maintenance in an emerging AI video modality, alongside its reported tests of 38 models.
AI 实时交互视频,是今年最热的技术方向之一。
过去大半年,相关模型密集破圈。阿里推出 HappyOyster,昆仑万维发布 Matrix-Game 4.0、爱诗科技发布 PixVerse R2,生数科技 7 月推出 Vidu S1,9 月紧接着发布 Vidu S2……
资本同样在跑步入场,多家公司完成新一轮融资,世界模型、数字人、实时编辑…… 各路人马从不同方向涌入。
但火热背后,这个赛道正处于一种混乱状态。
混乱首先来自定义。同样做实时交互视频,有人叫它「交互世界模型」,有人叫「实时交互多模态模型」,有人把它理解为独立于视频生成和世界模型之外的第三条路线。连赛道叫什么都没有共识,评测标准自然更谈不上。
现有评测工具还在沿用老框架,给模型一段提示词,生成一段视频,看画面清不清晰、动作合不合理、风格符不符合描述,VBench、EvalCrafter 等主流基准都是这个逻辑,但对实时交互视频系统来说,这套方法测的根本不是最难的那部分。
#StreamEval#Tsinghua University#NVIDIA#CMU#UC Berkeley#Princeton University#National University of Singapore#VBench#EvalCrafter#Vidu S2-Avatar#Vidu S2-Editing