Apple Silicon

Headlines · 7

1. 2h
   Kapitsa: open-source Apple Silicon inference engine tuned for Qwen and Nemotron

   Developer Ye at Nace.ai posted Kapitsa on r/LocalLLaMA, an Apache 2.0 open-source inference engine for Apple Silicon aimed at running coding agents on local models on Macs. Kapitsa supports only two model families — Qwen (3.5 4B, 3.8 27B, 3.6 35B-A3B and 3.8 Flash-Next) and NVIDIA Nemotron (3.5 Lightning, Bolt, Super-VL plus Dory 4B) — so the team can customize Metal kernels per architecture. The author published decode-speed tables on M1 Pro 16GB, M4 Pro 48GB and M5 Max 128GB machines, claiming it beats mlx-serve, oMLX, Ollama, MLX and llama.cpp in most cases, while trailing mlx-serve and oMLX by roughly 2-3% on Qwen3.8 Flash-Next. He also warns readers not to trust benchmarks, points them to a built-in kapitsa bench command to test themselves, notes that 4-bit quantization costs some quality, and says the new codebase should be expected to have bugs.

   r/LocalLLaMA · 🔥 5
2. 6h
   SGLang v0.5.22 ships new model support and up to 2.3x throughput gains

   SGLang has released v0.5.22, with its release notes stating the version covers 950 pull requests from 245 contributors. The release adds support for models including Clef / Clef Flash, PPLX-Decider v1 / v1.1, EmbeddingGemma 2, LLaDA2.2, HunyuanImage-3.0-Instruct and Kandinsky 6. The notes say DeepSeek-V4.1 gains up to 2.3x more throughput per GPU on H200 and now runs on AMD MI350X / MI355X with up to 1.9x throughput from DSpark, while GLM-5.3 can combine decode context parallelism with MTP for up to 2.1x faster decoding. The release also requires Python 3.11 or newer and deprecates options such as --enable-dp-attention, making it a breaking-change upgrade.

   SGLang Releases · 🔥 8
3. 3d
   Rust framework DenseCore claims bit-exact PyTorch match on nanoGPT

   A computer science student posted on r/MachineLearning about DenseCore, a small Rust deep learning framework with a PyTorch-like Python API that is roughly 2 MB versus about 580 MB for PyTorch. The author says that running Karpathy's nanoGPT unchanged against it produces identical initial weights from manual_seed, loss, gradients and updated weights bit for bit across all 50 tested steps, including with dropout enabled. Getting there required matching undocumented PyTorch behaviour such as the CPU flash attention kernel and its dropout path, rounding differences in Adam's tail loop, and clip_grad_norm_ using a reciprocal multiply; the match currently holds only on Apple Silicon Macs.

   r/MachineLearning · 🔥 0
4. 4d
   Apple MLX runs 816B Muse Spark Open locally on Mac Studio cluster

   Apple Machine Learning Research published a post on October 15, 2026, detailing how the latest MLX release runs the nearly trillion-parameter Muse Spark Open (816B total, 42B active) locally on Mac Studio. It says a single Mac Studio offers up to 512GB of unified memory and can scale across multiple Macs using an RDMA over Thunderbolt driver; on a cluster of four M5 Ultra Mac Studios, the model generated over 36 tokens per second at an 8,192-token context. Apple also compared M5 Ultra with M3 Ultra and benchmarked Mistral Medium 3.5, Qwen 3.8, and pipeline and tensor parallelization schemes, claiming up to 4.6x the start-up time-to-first-token speed of the previous Mac Studio generation.

   Apple Machine Learning Research · 🔥 0
5. 5d
   Show HN: Pulse Local runs budgeting AI fully on your Mac

   Developer Niz launched Pulse Local on Hacker News's Show HN, a beta budgeting app for Mac (Apple Silicon) that answers spending questions with models running on the machine. Users import a CSV from any bank in any currency; a fine-tuned Qwen3 0.6B model turns each question into a structured query, while code rather than the model does the money math, and a 4B checker model double-checks every money answer, with the app saying it is unsure when the two readings disagree. The author says this cut wrong answers from roughly one in five to 5 in 494 questions across the last three tests; local mode needs no account and sends no ledger data or analytics, but first start downloads about 2.9 GB of models and a money answer takes 12–14 seconds with the checker on. The app is free, with an optional Connected mode (Sync Pass, $29/year) that syncs US banks via Plaid, passing transactions through server memory without storing them.

   Hacker News · AI（100+ 分） · 🔥 0
6. Oct 12
   Apple's Minions turns idle Macs into an internal virtual GPU cloud

   Apple researchers published a paper describing Minions, an internal "virtual GPU cloud" that aggregates idle Apple Silicon Macs across the network to run machine learning inference. A Minion Server splits submitted jobs into subtasks stored in PostgreSQL, a Minion Executor on each Mac pulls subtasks over REST APIs and runs them on the local GPU with inputs and outputs in Amazon S3, and a Minion Watchdog requeues unfinished work when a machine sleeps, goes offline, powers off or its user stops the executor, using five-minute heartbeats. The paper says Minions only handles inference — embeddings with Hugging Face transformer models and text generation through Ollama installed on each Mac — and Apple reports that some jobs that once faced 10-plus-hour centralized GPU queues could start within minutes, figures that come from Apple's own testing and leave cross-team data confidentiality as an open concern.

   Apple Machine Learning Research · 🔥 0
7. Oct 8
   vLLM climbs GitHub Trending among Python projects

   vLLM is a high-throughput, memory-efficient inference and serving engine for large language models, originally developed in the Sky Computing Lab at UC Berkeley and now maintained by a community of more than 2,000 contributors. It manages attention key-value memory with PagedAttention and supports continuous batching, chunked prefill, prefix caching, speculative decoding and quantization formats including FP8, INT8/INT4, GPTQ/AWQ and GGUF, alongside an OpenAI-compatible API server. The engine runs on NVIDIA, AMD and Intel GPUs plus x86/ARM/PowerPC CPUs, and supports 200+ model architectures on Hugging Face.

   GitHub Trending · Python · 🔥 0

Experience and discussion from the community

Share my Apple Silicon experienceAsk about Apple Silicon