Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Alibaba MNN opens MNN-Pocket-0.3B for on-device AI

AI summary

Alibaba’s MNN team open-sourced MNN-Pocket-0.3B, a roughly 300-million-parameter language model designed for Chinese and English mobile applications. The team reports that its INT4 deployment is 173MB, runs fully offline on an iPhone 18 Pro at about 350 tokens per second, and reached 466 tokens per second in an iPad Pro M5 Metal test. It also released Base, SFT, MNN, DFlash2, and decision-model variants covering pretraining, post-training, deployment, speculative decoding, and candidate selection.

In-depth insight

The release provides a compact, openly available model and deployment stack built around MNN for integrating generative features into resource-constrained mobile apps.

IT之家 11 月 6 日消息,淘宝旗下 MNN 团队今日发布了 MNN-Pocket-0.3B 模型,面向手机应用的任务需求与资源预算,在有限算力、内存和能耗下,兼顾任务能力、响应速度与持续运行体验。

围绕淘宝等综合型手机应用,团队将“手机应用原生大模型”落实为四个维度的设计目标:

  • 速度上,在指定设备与上下文配置下,解码吞吐超过 200token/s。这一目标参考高速云端模型服务的生成吞吐,力求在本地提供流畅的生成体验。
  • 内存上,在指定测试条件下,峰值内存低于 300MB。即使手机拥有 8~16GB 内存,也需要由系统与多个应用共享。以约 500MB 的模型模块预算为例,将推理峰值控制在 300MB 以下,可以为上下文增长和集成开销保留余量。
  • 功耗上,在典型推理负载(输入 512~1024token、输出 128token)下,单次推理增量能耗目标不超过参考手机电池能量的万分之一,即 100 次推理耗能不超过电池能总量的 1%,控制高频调用的续航开销。
  • 能力上,以在指定业务场景上达到 8B 级模型的效果为目标,兼顾中英文理解、世界知识与数学、代码推理能力,重点强化指令遵循,并为长上下文与多步骤任务保留扩展空间。

#Alibaba#MNN#MNN-Pocket-0.3B#MNN-Pocket-0.3B-Base#MNN-Pocket-0.3B-SFT#MNN-Pocket-0.3B-MNN#MNN-Pocket-0.3B-DFlash2#MNN-Pocket-0.3B-Dec#MNN inference engine#Qwen3-0.6B#MiniCPM4-0.5B

Original · 5 min readRead the original
Group chat

No comments yet. Start the conversation.