Private Mind 支持离线运行 LFM3
AI 总结
开源应用 Private Mind 的开发者称，该应用现可在 iOS 和 Android 上完全离线运行 LFM3，包括视觉模型 LFM3-VL-2B；开发者还发布了 LFM3 的 ExecuTorch 导出。其 iPhone 16 Pro 实测称，4-bit LFM3-1.7B 约为每秒 40 个 token，而 LFM3-VL-2B 在首个 token 约需 9 秒后，解码速度约为每秒 20 个 token。导出构建分别采用面向 Apple GPU 的 MLX 和面向移动 CPU 的 XNNPACK，报告的模型大小分别为 1.07 GB 和 1.17 GB。
深度见解
此次更新提供了在移动硬件上离线运行文本与视觉模型的开源路径，并同时给出导出的运行时构建和设备实测数据。
| LFM3 is now in Private Mind, our open source app for running models locally on iOS and Android. In the video, LFM3-VL-2B reads a Japanese ramen menu: asked what to order without spice, it skips the bowl marked 激辛 (extra spicy) and picks the miso ramen; asked about paying by card, it finds the "cash only" line under the prices. Numbers on an iPhone 16 Pro: Exporting LFM3 to ExecuTorch LFM3 isn't LFM2 with more parameters, and ExecuTorch's existing LFM2 export doesn't cover it. The new pieces: So I rewrote the decoder in plain PyTorch for export and checked it against Liquid's reference implementation: max logit difference 0.00005, same top 10 tokens. The part I liked most is the sliding-window cache. ExecuTorch's built-in ring-buffer cache is sized at 2x the window, which caps a single forward call at 512 tokens, so a long prompt has to be fed in small pieces. I gave the sliding layers a buffer of window + 2048 and let the attention mask enforce the 256-token window, so one call can take 2048 tokens and the context still goes to 8192. Two builds came out of it: MLX 4-bit for the Apple GPU (1.07 GB) and XNNPACK 8da4w for CPU on iOS and Android (1.17 GB). The LFM3 exports are on Hugging Face: https://huggingface.co/software-mansion/LFM3-ExecuTorch submitted by /u/K4anan [link] [comments] |
#Private Mind#LFM3#LFM3-1.7B#LFM3-VL-2B#ExecuTorch#MLX#XNNPACK#Software Mansion#Hugging Face#Apple