XNNPACK

Headlines · 3

1. 2d
   Private Mind adds offline LFM3 inference on iOS and Android

   The developer of the open-source Private Mind app reports that it now runs LFM3 models fully offline on iOS and Android, including the LFM3-VL-2B vision model. They also published ExecuTorch exports for LFM3 and reported iPhone 16 Pro results of about 40 tokens per second for 4-bit LFM3-1.7B, versus about 20 tokens per second for LFM3-VL-2B after an approximately nine-second first-token delay. The exports use MLX for Apple GPUs and XNNPACK for mobile CPUs, with reported model sizes of 1.07 GB and 1.17 GB respectively.

   r/LocalLLaMA · 🔥 2
2. 5d
   Nemotron 3.5 Streaming ASR Benchmarked on Pixel 11

   PyTorch profiled multilingual Nemotron 3.5 streaming ASR through ExecuTorch on a Google Pixel 11, testing chunk sizes, FP16 and QC8W precision, and Arm SME2 acceleration. With SME2 enabled, processing a 24.9-second recording fell from 9.79 to 4.61 seconds for FP16 and from 5.20 to 3.24 seconds for QC8W. The study also shows the accuracy-latency tradeoff: longer audio chunks improved word error rate but increased estimated time to first token.

   PyTorch Blog · 🔥 0
3. Oct 19
   Google Releases LiteRT Tensor API for Native Edge Model Authoring

   Google announced the LiteRT Tensor API, a C++ authoring layer that lets developers construct computation graphs operation by operator for on-device AI. Google says the same graph can run through LiteRT on CPUs, ML Drift-powered GPUs and NPUs, or directly through XNNPACK on CPU, with reference implementations including SAM 2 and Gemma models.

   Google Developers Blog · 🔥 0

Experience and discussion from the community

Share my XNNPACK experienceAsk about XNNPACK