MNN inference engine
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Alibaba MNN opens MNN-Pocket-0.3B for on-device AI
Alibaba’s MNN team open-sourced MNN-Pocket-0.3B, a roughly 300-million-parameter language model designed for Chinese and English mobile applications. The team reports that its INT4 deployment is 173MB, runs fully offline on an iPhone 18 Pro at about 350 tokens per second, and reached 466 tokens per second in an iPad Pro M5 Metal test. It also released Base, SFT, MNN, DFlash2, and decision-model variants covering pretraining, post-training, deployment, speculative decoding, and candidate selection.
IT之家 AI · 🔥 0
Experience and discussion from the community
Share my MNN inference engine experienceAsk about MNN inference engine
Nobody has shared their experience with MNN inference engine yet.