Apple Neural Engine

Headlines · 2

1. Oct 24
   Forge 0.2.1 Adds 192K-Context Qwen3.8-27B Support on M6

   According to the developer’s report, ANEMLL Forge 0.2.1 can run Qwen3.8-27B on the Neural Engine of a 32GB M6 Mac mini with context windows up to 192K tokens. The release adds an INT8 K/V cache and a smaller drafter that shares the target model’s output head, reportedly saving 0.86GB of ANE memory without reducing speed. The developer reports content-dependent decode rates of 10–62 tokens per second, a 96K configuration using about 21.4GB, and an initial full 192K compilation time of roughly two hours.

   r/LocalLLaMA · 🔥 0
2. Oct 19
   Developer launches Tinkerbox to run decision models on Apple's Neural Engine via Shortcuts

   A developer posting on r/LocalLLaMA as /u/WeGoToMars7 released Tinkerbox, an app that runs decision models on Apple's Neural Engine (ANE) and exposes Shortcuts actions for automating system events such as incoming email and iMessages, folder and file changes, time of day and alarms. The author says LiquidAI d1-3B runs on the ANE with roughly 40 ms latency for warm short-context decisions under 256 tokens, measured on an M5 Pro, and says the model sits on the Pareto frontier of Decision Index 0.3.2; both are self-reported or cited by the developer, who notes he has no other Macs to test on. The d1-3B implementation uses a tree structure to evaluate multiple decisions in a single pass, source code and AOT-compiled models are published on GitHub and Hugging Face, and the author says he is working on integrating vLLM d3 models based on Qwen3.5 with 0.8B, 2B and 4B variants planned.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my Apple Neural Engine experienceAsk about Apple Neural Engine