Metal

Headlines · 3

1. Oct 22
   Kapitsa adds Flint support, claims SOTA Apple Silicon decode

   A contributor to Kapitsa, an open-source inference engine written with custom Metal kernels and targeted at Apple Silicon from M1 through M6, posted that the project (version 0.2.7) has added support for Recursive's newly released Flint model, which Recursive says is optimized for coding on Macs with 32GB or more and beats Qwen's 35B sparse MoEs while nearing Qwen's 27B dense model. On a single MacBook Pro M4 Pro with 48GB, the contributor's own benchmarks put Kapitsa at 127.9 tok/s short-context decode versus 126.6 for Recursive's rec 0.1.0 and 79.6 for llama.cpp b11804, with Kapitsa's lead over rec growing from 1.06x at 8K context to 1.30x at 128K. The author notes only one Mac was tested, that Flint has not been run on an M5, that prefill on M5+ is not yet optimized, and that a KFlash tree drafter trained on eight H200s for Flint Q4, claimed to give an internal 1.4x decode speedup without accuracy loss, is due Monday; running it requires a Mac with at least 32GB and a 22.6GB download.

   r/LocalLLaMA · 🔥 0
2. Oct 20
   PyTorch 2.15 MPS adds FP8 and faster LLM decoding on Macs

   Hugging Face's PyTorch team published a blog post on October 20, 2026 describing MPS improvements in PyTorch 2.15, including FP8 support with float8_e4m3fn storage and scaled matrix multiplication on the Mac backend. The team reports self-measured gains: MoE decoding 5–6.5× faster than 2.14, with Qwen3.6-35B-A3B rising from 9.4 to 61.3 tokens/s on an M3 Ultra, and Wan2.2 video VAE decode peak memory falling from 270 GiB to about 8 GiB. The post also says MPS now requires macOS 15 or later, PyTorch 2.15 requires Python 3.11 or later, and 2.16 will run attention entirely on Metal.

   Hugging Face · 🔥 0
3. Oct 8
   Google open-sources ML Drift GPU inference engine for on-device AI

   Google's AI Edge team announced on October 8 the open-source release of ML Drift, an Apache 2.0-licensed, cross-platform GPU compute engine built for on-device AI/ML inference. The engine abstracts low-level on-device GPU APIs including OpenGL ES, OpenCL, Metal and WebGPU, and serves both as the core GPU acceleration engine inside LiteRT and as a standalone library. Google says ML Drift already runs in production across Chrome, YouTube Shorts, Photos and Meet; it reports up to 12% lower memory overhead than other frameworks in Gemma benchmarks and up to 40% lower average frame latency for Shorts segmentation effects, both vendor claims. Desktop previews on Windows and Linux run through Dawn, Chromium's WebGPU implementation.

   Google Developers Blog · 🔥 0

Experience and discussion from the community

Share my Metal experienceAsk about Metal