VibeVideoBench评测代码驱动视频生成
论文作者推出 VibeVideoBench,包含239项中英文任务,覆盖12个类别,用于评估通过编写代码而非调用图像或视频生成模型制作视频的多模态智能体。每项任务要求智能体在一小时内交付带声音的完整MP4,评测结合交付检查、约1.3万项任务规则检查、任务特定质量标准及三组VLM评委。测试覆盖9个智能体系统和2,151次计分运行,GPT-6 Astra与Claude Opus 5.5在评委和人类标注者排名中位列前两名,最强系统满足全部核心要求的任务占比为76.3%。
该基准为衡量多模态智能体通过代码交付完整视听作品的能力提供了细化方法,并将交付失败单独列为重要维度。
Computer Science > Computer Vision and Pattern Recognition
arXiv:2610.35864 (cs)
[Submitted on 26 Oct 2026 (v1), last revised 27 Oct 2026 (this version, v2)]
Title:VibeVideoBench: Evaluating Multimodal Agents on Code-Driven Video Generation
Authors:Muxi Diao, Wenbin Xu, Lele Yang, Xuyang Ma, Yuzhe Chen, Jiankang Chen, Kaiyu Jiang, Changyi Liu, Haoyu Zhang, Hongyi Fu, Jianxiong Wang, Tianke Zhang, Haonan Fan, Yingxin Li, Bingyi Li, Xu Wang, Xuanyu Zheng, Bin Wen, Tingting Gao, Han Li, Wentao Zhang, Weiran Xu, Kongming Liang, Zhanyu Ma
View a PDF of the paper titled VibeVideoBench: Evaluating Multimodal Agents on Code-Driven Video Generation, by Muxi Diao and 23 other authors