Multimodal Flow Unifies Language and Vision With Continuous Modeling
Huazhong University of Science and Technology, Horizon Robotics, and other collaborators introduced Multimodal Flow, a fully continuous architecture that models language and vision in their native embedding spaces under one Flow Matching objective. Its 1.6B-parameter MF-1 model was pretrained from scratch on 150B tokens and, according to the report, outperformed Transfusion-style and Chameleon-style baselines under matched data, parameter, and optimization budgets. The project includes a paper, source code, and project page.
Multimodal Flow offers a third route between fully discrete tokenization and hybrid discrete-continuous systems, using one continuous generation objective for multiple multimodal tasks.
机器之心 2026-11-05 16:21 北京
连续统一多模态模型的想象空间,或许才刚刚打开。
走向物理 AI 时代,构建一个能真正统一语言与视觉的多模态基座模型是关键一步。然而,语言本质上是有序的 Token 序列,视觉则是连续的高维像素分布,两者在信息结构上存在天然的鸿沟。如何在一个模型内统一这两种截然不同的分布,一直是一个备受争论的话题。
目前,主流的解法主要有两条。一条是全离散路线(如 Chameleon、Emu3),它把图像也量化成离散 Token,与语言拼在一起做自回归建模。但这种方法受限于视觉 Tokenizer,图像的高频细节在量化过程中极易丢失。另一条是混合离散 - 连续路线(如 Transfusion、JanusFlow),文本走离散预测,图像走连续扩散。虽然避开了量化瓶颈,但一个模型里得同时维护两套截然不同的目标函数和采样流程,不仅工程实现复杂,联合优化的难度也很大。
#Multimodal Flow#MF-1#Huazhong University of Science and Technology#Horizon Robotics#SigLIP2#T5-small#Chameleon#Emu3#Transfusion#JanusFlow