FramePack-U packs hour-long video into a fixed token budget at constant memory
The paper "Elephant in the Fridge: Constant-Memory Frame Packing for Long Video Understanding" introduces FramePack-U, a training-free frame-packing framework that compresses arbitrarily long videos into a fixed-size visual representation at constant memory cost, requiring no fine-tuning or architectural change and plugging directly into off-the-shelf vision-language models (VLMs) at inference time. It scores each frame by semantic similarity to the text prompt, ranks frames in a diversity-driven manner, and packs the selected frames at varying resolutions into a fixed-length context window so the memory budget goes to frames that are both instruction-relevant and visually diverse. The authors report that on LongVideoBench, MLVU and Video-MME, FramePack-U consistently beats full-resolution baselines and prior token-reduction and keyframe-selection methods, including gains of +12.9%, +10.6% and +9.7% over the official model on Qwen3.5-35B-A3B with a 128-frame budget.
It targets the linear memory and latency growth that limits long-video understanding, and works with existing VLMs without retraining, which could lower the bar for deploying long-video inference.