开发者推出 Tinkerbox：在苹果神经引擎上跑决策模型
AI 总结
开发者 /u/WeGoToMars7 在 r/LocalLLaMA 发布 Tinkerbox 应用，可在 Apple 神经引擎（ANE）上运行决策模型，并提供 Shortcuts 动作，用于来电邮件、iMessage、文件夹与文件变动、时间与闹钟等系统事件触发的自动化。作者称 LiquidAI d1-3B 在 M5 Pro 上对短上下文（少于 256 token）的热决策延迟约为 40 毫秒，并称该模型目前位于 Decision Index 0.3.2 的帕累托前沿；这些均为其自行测量与引用，他本人表示没有其他 Mac 可供验证。据其介绍，Tinkerbox 的 d1-3B 实现用树结构在一次前向中评估多个决策以降低延迟，源代码与 AOT 编译模型已发布在 GitHub 与 Hugging Face；作者还表示正在集成基于 Qwen3.5 的 vLLM d3 模型，计划推出 0.8B、2B 与 4B 版本。
深度见解
作者认为决策模型适合 ANE，因为它们不做逐 token 自回归解码，而 ANE 因内存带宽较低和静态缓冲区限制在这一环节表现不佳，这为该硬件上运行低延迟决策类模型提供了一条路径。
| LiquidAI d1-3B is currently on the Pareto frontier of the Decision Index 0.3.2 https://huggingface.co/spaces/multimodalart/jev-decision-index, and you can run it on the ANE with around 40 ms latency on warm short-context (<256 tokens) decisions, as measured on M5 Pro. With Shortcuts integration, you can run efficient decisions on events such as incoming email and iMessages, folder and file modifications, time of day and alarms, and other triggers without writing a single line of code. Decision models are a good fit for ANE because they don't use token-by-token autoregressive decoding, which it struggles with due to relatively low memory bandwidth and static buffer limitations. Tinkerbox's implementation of d1-3B uses a tree structure to evaluate multiple decisions in a single pass, further reducing latency. Source code: https://github.com/WeGoToMars/tinkerbox AOT-compiled models: https://huggingface.co/wegotomars/d1-3B-CoreAI-ANE Download the app: https://github.com/WeGoToMars/tinkerbox/releases I don't have access to any other Macs, so please report your results and any issues you encounter. I'm also taking model and feature suggestions! Embeddings, TTS, STT, and image segmentation models are on the roadmap. This isn't a one-off vibecoded repo, I want to continue supporting it and make it a real framework for ANE-optimized inference. I'm currently working on integrating vLLM d3 models (based on Qwen3.5), which would also allow you to use most other Qwen3.5 fine-tunes with minimal modification. 0.8B (lite), 2B (nano), and 4B (mini) should be available soon! submitted by /u/WeGoToMars7 [link] [comments] |
#Tinkerbox#Apple Neural Engine#LiquidAI#LiquidAI d1-3B#Apple Shortcuts#Qwen3.5#Apple