Private Mind adds offline LFM3 inference on iOS and Android
The developer of the open-source Private Mind app reports that it now runs LFM3 models fully offline on iOS and Android, including the LFM3-VL-2B vision model. They also published ExecuTorch exports for LFM3 and reported iPhone 16 Pro results of about 40 tokens per second for 4-bit LFM3-1.7B, versus about 20 tokens per second for LFM3-VL-2B after an approximately nine-second first-token delay. The exports use MLX for Apple GPUs and XNNPACK for mobile CPUs, with reported model sizes of 1.07 GB and 1.17 GB respectively.
The update provides a concrete open-source path for running text and vision models locally on mobile hardware, together with exported runtime builds and device measurements.
| LFM3 is now in Private Mind, our open source app for running models locally on iOS and Android. In the video, LFM3-VL-2B reads a Japanese ramen menu: asked what to order without spice, it skips the bowl marked 激辛 (extra spicy) and picks the miso ramen; asked about paying by card, it finds the "cash only" line under the prices. Numbers on an iPhone 16 Pro: Exporting LFM3 to ExecuTorch LFM3 isn't LFM2 with more parameters, and ExecuTorch's existing LFM2 export doesn't cover it. The new pieces: So I rewrote the decoder in plain PyTorch for export and checked it against Liquid's reference implementation: max logit difference 0.00005, same top 10 tokens. The part I liked most is the sliding-window cache. ExecuTorch's built-in ring-buffer cache is sized at 2x the window, which caps a single forward call at 512 tokens, so a long prompt has to be fed in small pieces. I gave the sliding layers a buffer of window + 2048 and let the attention mask enforce the 256-token window, so one call can take 2048 tokens and the context still goes to 8192. Two builds came out of it: MLX 4-bit for the Apple GPU (1.07 GB) and XNNPACK 8da4w for CPU on iOS and Android (1.17 GB). The LFM3 exports are on Hugging Face: https://huggingface.co/software-mansion/LFM3-ExecuTorch submitted by /u/K4anan [link] [comments] |
#Private Mind#LFM3#LFM3-1.7B#LFM3-VL-2B#ExecuTorch#MLX#XNNPACK#Software Mansion#Hugging Face#Apple