ModelBuilder says optimized MiniCPM-V 4.7 35B-A3B exceeds 30 tokens/s on phones
At a ModelBuilder (面壁智能) on-device multimodal model event, co-chief scientist Yao Yuan said a newly optimized MiniCPM-V 4.7 35B-A3B can exceed 30 tokens/s on phones while using about 3GB of memory, with almost no accuracy loss. Zhidx measured about 25 tokens/s while testing the model on an iPhone Duo, below the vendor's figure. Earlier team data showed speeds above 13 tokens/s on an iPhone 17 Pro Max and above 17 tokens/s on an iPad Pro M4 16GB. ModelBuilder released and open-sourced the MiniCPM-V 4.7 series on October 13, including 35B-A3B and 1B versions. Its Lookahead Expert Scheduling (LES) predicts which experts the next layer will need and loads their weights from flash storage while the current layer computes. The company says LES preserves model output and increased generation speed by about fivefold on an iPad Pro M4 16GB.
The reported results show that a 35-billion-parameter multimodal MoE model can approach interactive generation speeds on a phone with about 3GB of memory, while LES addresses a key bottleneck in running MoE models within limited device memory. However, the independently observed speed was about 25 tokens/s, and ModelBuilder said the model's power consumption is close to major mobile games and that all-day continuous operation is not currently realistic.

智东西 作者 | 江宇 编辑 | 云鹏
350亿参数的多模态大模型,在仅占用约3GB内存的情况下,已经能在手机上实现超过30 Token/s的生成速度。
智东西11月5日报道,今日,在面壁智能举行的端侧多模态大模型技术交流活动上,面壁智能联席首席科学家、清华大学人工智能学院助理教授姚远透露,团队通过最新一轮推理优化,已将MiniCPM-V 4.7 35B-A3B在手机上的生成速度进一步提升至30 Token/s以上,模型精度损失几乎可以忽略。
一款总参数量达到350亿、每次推理仅激活约30亿参数的混合专家(MoE)模型,正在接近智能手机上较为流畅的实时交互速度。
此前,面壁智能已于10月13日发布并开源MiniCPM-V 4.7系列模型,包括35B-A3B和1B两个版本。
#ModelBuilder (面壁智能)#MiniCPM-V 4.7 35B-A3B#MiniCPM-V 4.7 1B#Lookahead Expert Scheduling (LES)#LLaVA-UHD v4#DSpark#iPhone 17 Pro Max#iPad Pro M4#iPhone Duo#Apple Intelligence#RLPR#Self-Replay Fine-Tuning