VideoMemBench tests memory in multi-turn video conversations
VideoMemBench is a new benchmark for evaluating whether multimodal large language models can retrieve and reuse visual information and earlier responses across a long, multi-turn conversation about one video. It contains 326 videos and 2,476 turns, including 1,387 turns with cross-turn dependencies, and its dataset and evaluation have been released. In author-reported tests of more than 20 models, Claude Opus 5 ranked first with an overall score of 45.7%.
The benchmark addresses a capability not measured by conventional video question-answering tests: maintaining in-context video memory over extended dialogue.
Computer Science > Computer Vision and Pattern Recognition
arXiv:2610.40131 (cs)
[Submitted on 29 Oct 2026 (v1), last revised 3 Nov 2026 (this version, v3)]
Title:VideoMemBench: How Well Do MLLMs Remember in Video Conversations?
Authors:Zhengdong Hu, Chao Wang, Kaituo Feng, Huanjin Yao, Mingfei Lu, Hehe Fan, Yi Yang
View a PDF of the paper titled VideoMemBench: How Well Do MLLMs Remember in Video Conversations?, by Zhengdong Hu and 6 other authors