Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

MLX

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 33

  1. Hugging Face ships Inkling 2 Mini, a looped-transformer MoE for local setups

    Hugging Face announced that Thinking Machines Lab's Inkling 2 Mini is available, describing it as a multimodal mixture-of-experts model with about 26B physical total parameters and 3B active parameters per token. Its 24 decoder layers include 12 middle layers that run three times in a loop, giving it the effect of 48 logical layer visits, and it supports a 1M-token context with image and text inputs. Weights ship in mixed precision with NVFP4 and MXFP8 expert weights, a DSpark drafter is included, and support is available in transformers, Llama App, llama.cpp, MLX and SGLang with day-zero inference. Hugging Face says in its informal vibe tests the model was at least comparable with Qwen 3.8 27B, while noting that more validation is needed.

    Hacker News 首页(100+ 分) · 🔥 39
  2. Ollama v0.40.17 improves CLI recovery and MLX concurrency

    Ollama v0.40.17 adds improved ChatGPT CLI restore and graceful restart, along with support for using thinking metadata in OpenCode settings. The release also introduces memory mapping for Vulkan integrated GPUs, concurrent generation in MLX, and version bumps for llama.cpp, MLX, and MLX-C. Ollama's release notes identify these changes under v0.40.17, while the listed changelog endpoint ends at v0.40.17-rc0.

    Ollama Releases · 🔥 3
  3. Muse Spark Open brings large-model AI to Apple silicon Macs

    9to5Mac reports that Meta released Muse Spark Open, an 816-billion-parameter open-weight model with 42 billion active parameters and a million-token context window, optimized for Mac Studio. The column says Apple’s MLX framework and unified memory let a pair of M5 Ultra Mac Studios run the model, while Osaurus can run usable offline models on 16 GB Apple silicon laptops and offers two models for 8 GB systems.

    9to5Mac · 🔥 0
  4. Nace releases open-source KQuant MLX packages for Qwen models

    Nace released free, open-source KQuant packages in standard MLX format for Qwen3.5-4B, Qwen3.5-9B, Qwen3.8-27B, Qwen3.6-35B-A3B, and Qwen3.8-Flash-Next. In the developers’ measurements, the packages had KL divergence from full-precision next-token predictions up to 3.7 times lower than cited public 4-bit baselines. On an M5 Pro, selected models were reported to write 6%–15% faster with Kapitsa 0.2.35 and the KFlash drafter, although baseline tests used earlier Kapitsa versions. The packages require no custom kernels or special loader. GGUF versions were still being tested ahead of a planned public release.

    r/LocalLLaMA · 🔥 0
  5. mlxVS brings cuVS-style GPU vector search to Apple Silicon

    The developer released mlxVS, an Apache-2.0 library built on MLX that ports NVIDIA cuVS-style exact k-NN, k-means, NN-Descent, and CAGRA algorithms to Apple Silicon through Python and C++ interfaces. On SIFT1M and a MacBook Pro M3 Pro, the developer reported CAGRA recall@10 of 0.976 at 69,500 queries per second with itopk 64, and 0.993 at 37,800 queries per second with itopk 128. These figures apply to batched queries, were not independently reproduced, and lack an apples-to-apples comparison with CUDA-based cuVS.

    r/MachineLearning · 🔥 0
  6. M5 Ultra Mac Studio beats a single GB10 in agentic test but trails a Spark pair

    Tom's Hardware benchmarked an M5 Ultra Mac Studio with a 36-core CPU, 80-core GPU and up to 512GB of unified memory against a single Dell Pro Precision GB10 system, which it described as nearly identical to an Nvidia DGX Spark. Using llama.cpp with Qwen 3.8 27B and its native multi-token prediction, the Mac completed the Artificial Analysis AgentPerf-Local test in just under half the time of the GB10 system. The report also cited 1.2 TB/s of memory bandwidth for the Mac versus 273 GB/s for the GB10. For DeepSeek V4.1 Flash 1017, however, the Mac's llama.cpp setup took 10–25 minutes for the longest tested contexts, while a two-Spark cluster using DSpark speculative decoding and TensorFold took under five minutes. The report said Mac Studio lead times stretched into February 2027.

    Tom’s Hardware · 🔥 0
  7. mlx-decision Adds LoRA Image Fine-Tuning for Doom on Mac

    Developer armbues reports adding image-based LoRA fine-tuning to the open-source mlx-decision tool, enabling its 2B decision model to process Doom frames on Apple Silicon. The author says training on about 3,000 ViZDoom frames took 43 minutes on an M5 Pro, while held-out-frame metrics improved and the model averaged 23.2 kills across 20 episodes. The MIT-licensed project targets macOS 14 or later, Python 3.12, and Apple Silicon.

    r/LocalLLaMA · 🔥 0
  8. 35 Qwen MoE domain LoRAs released with mixed results

    A Reddit author released 35 domain-expert LoRA adapters for Qwen3.6-35B-A3B under Apache-2.0, along with datasets, domain configurations, a MiniLM router, and MLX training scripts. The adapters were trained in BF16 on one 512GB Mac Studio M3 Ultra; 8-bit fusion produced unusable output, while runtime BF16 hot-swapping reportedly took about two seconds per request. On the author's evaluations, routing reached about 46% top-1 and 70% top-3 accuracy, and the base model outperformed most specialist adapters except in HTML/CSS.

    r/LocalLLaMA · 🔥 0
  9. Ollama v0.40.16 adds Ripple Preview to its MLX runner

    Ollama v0.40.16 added Ripple Preview support to its MLX runner, allowing users to launch Ricursive Intelligence's 65B dense model for agentic coding with `ollama run ripple-preview`. The release also reports that Qwen3.8 Flash Next uses about 27 GiB less memory on Apple silicon, though no independent measurement was supplied. Other changes fix Claude Code routing, clarify retired cloud-model errors, forward truncation options for the OpenAI Responses API, and return an empty array from `/v1/models` when no models are installed.

    Ollama Releases · 🔥 0
  10. Developer details Lamplight, a local-first AI harness running two machines

    A developer detailed Lamplight, a custom harness built on Pi-Bolt plus custom extensions that links two machines — a MacBook Pro (M5 Max, 128 GB unified memory) and a self-built small-form-factor PC (RTX 5090 FE, Ryzen 7 9800X3D, 64 GB RAM) — so one can orchestrate the other via git worktrees and an event-driven job system, with Claude also involved. He says roughly 95% of his AI use now runs locally, with both machines running Qwen 3.8 Flash-Next at a reported 262K context, averaging 302 tok/s on the RTX 5090 machine (Basalt, IQ3_S) and 100 tok/s on the Mac (mlx-serve, 4.7bpw MLX), with HumanEval scores of 99.4%/93.9% and 98.8%/94.5%, LiveCodeBench v6 of 85.7% and 85.1%, and MMLU-Pro of 87.9% and 90.7%. He reports that undervolting the 5090 with a custom fan curve (575 W, 83 °C down to 471 W, 70 °C) and capping the Ryzen clock at 4.8 GHz during decode (120 W, 96 °C down to 77 W, 70 °C) fixed thermal problems, and that a "medium" reasoning setting on the Mac avoided output-cap stalls. The harness itself is not being shared yet, though the developer says he is decoupling parts of it to release components later.

    r/LocalLLaMA · 🔥 0
  11. Google Research releases TimesFM 3.0 forecasting model

    Google Research released TimesFM 3.0 with native univariate and multivariate time-series forecasting and support for past-only and past-and-future dynamic covariates. A PyTorch checkpoint is available as google/timesfm-3.0-pytorch, while the repository also describes an MLX backend for Apple silicon. The project claims leading results on three time-series benchmarks, but those claims are not independently verified in the supplied report.

    GitHub Trending · Python · 🔥 0
  12. iQuantize launches open-source desktop workspace for LLM quantization

    Developer Fazmin introduced iQuantize, an Apache 2.0 desktop workspace for quantizing large language models into GGUF, MLX, and ONNX formats. According to the Reddit post, users can load a local Safetensors model or select one from Hugging Face, follow a step-by-step conversion, compare the output with the original, and publish it with a model card.

    r/LocalLLaMA · 🔥 0
  13. Apple’s First Touchscreen MacBook Pro Pairs PROLED With Local AI

    爱范儿 reports that Apple introduced a touchscreen MacBook Pro with a PROLED XDR display, starting at 22,999 yuan. The new 14-inch and 16-inch models are lighter, feature an illuminated Apple logo, and retain keyboard and trackpad as the primary controls. In a hands-on test of a high-end 16-inch M5 Max configuration, 爱范儿 reported that Qwen 3.5 35B-A3B generated at 119–121 tokens per second under Ollama and MLX.

    爱范儿 · 🔥 0
  14. GPT-6.1-sol builds an AI-guided 3D vending machine

    A Juejin author says GPT-6.1-sol helped build a Three.js vending machine that stores 60 simulated drinks, dispenses cans, supports reversible disassembly, and provides guided camera movements. The current version uses Ming-3.0-tts for locally generated Chinese narration, including selectable voices, while an approximately 4.58 MB single-file edition preserves purchasing, disassembly, camera, and audio functions offline. The published recording predates the switch to Ming, so it does not demonstrate the current narration version's complete natural playback.

    掘金 · 人工智能 · 🔥 0
  15. Ollama v0.40.15 adds MiMo V2.6 support and Linux MLX CUDA loading

    Ollama’s v0.40.15 release notes list support for the MiMo V2.6 model and a change to load MLX’s bundled CUDA libraries in-process on Linux. Separate v0.40.15 release-candidate reports document an llama.cpp bump to b12111 and the removal of a patch superseded by an upstream fix in rc1, followed by Kimi Code CLI support in cmd/launch in rc2. The supplied reports do not establish whether those release-candidate changes were included in the final v0.40.15 build.

    Ollama Releases · 🔥 0
  16. Olla v0.1.0 adds TEI support and reliability improvements

    The Olla project released v0.1.0, its 25th release. The update adds Hugging Face Text Embeddings Inference support for embeddings, reranking and classification; configurable retry and failover behavior; a reworked dashboard; improved model-capability detection; and fixes for concurrency, request statistics, rate limiting, routing, backend discovery and configuration validation. Olla is a Go-based open-source proxy and load balancer that sits between applications and local inference servers. It supports multiple runtimes, including Ollama, vLLM, SGLang, llama.cpp and LM Studio, and provides OpenAI- and Anthropic-compatible interfaces with translation where required.

    r/LocalLLaMA · 🔥 0
  17. Ollama v0.40.14 updates Claude Code limits and API behavior

    According to Ollama's release notes, v0.40.14 makes Claude Code launches apply each model's output-token limit and adds a ChatGPT routing check to Codex configuration. The release also fixes decoding for cloud web-search responses on /api/chat, preserves readable input on the Responses endpoint, retains packaged sampler defaults on OpenAI-compatible endpoints, and allows the Ollama app to retry ChatGPT disconnects. Ollama also reports faster top-k sampling on MLX for large vocabularies.

    Ollama Releases · 🔥 0
  18. Private Mind adds offline LFM3 inference on iOS and Android

    The developer of the open-source Private Mind app reports that it now runs LFM3 models fully offline on iOS and Android, including the LFM3-VL-2B vision model. They also published ExecuTorch exports for LFM3 and reported iPhone 16 Pro results of about 40 tokens per second for 4-bit LFM3-1.7B, versus about 20 tokens per second for LFM3-VL-2B after an approximately nine-second first-token delay. The exports use MLX for Apple GPUs and XNNPACK for mobile CPUs, with reported model sizes of 1.07 GB and 1.17 GB respectively.

    r/LocalLLaMA · 🔥 0
  19. Mattt releases Swift packages for on-device speech and decisions

    Mattt announced AnySpeechRecognizer and AnyDecisionModel, two Swift packages for Apple platforms. AnySpeechRecognizer mirrors Apple’s speech-to-text API, supports selectable engines such as NVIDIA Parakeet and Alibaba Qwen3-ASR running locally through Core ML, and adds speaker diarization. AnyDecisionModel returns typed answers and probabilities for yes-or-no, multiple-choice, and rating-scale questions, using on-device models through MLX, Core AI, or llama.cpp, or decision models from Jev or Ollama. The packages can download open models from Hugging Face. Reported speed, memory, and word-error-rate results came from one machine and a few datasets; testing was limited to macOS 27, with no iPhone results.

    Hugging Face · 🔥 0
  20. MiniMax releases MiniMax-M2 for coding and agentic workflows

    MiniMax released MiniMax-M2, a mixture-of-experts model with 230 billion total parameters and 10 billion active parameters, and made its weights available for local deployment. The model is designed for coding and agentic workflows. MiniMax recommends SGLang, vLLM, MLX and Transformers for serving it, while explicitly citing day-zero support for SGLang and vLLM. The company also reported benchmark and efficiency results, but these claims remain vendor-reported. The model card lists a modified MIT license, while the release is described as open-source.

    MiniMax · Hugging Face · 🔥 0
  21. Study tests small open-weight models on real coding-agent work

    A retrospective empirical systems paper submitted to arXiv examines whether small open-weight models can complete real coding work rather than merely generate coherent text. Based on 4,165 gateway turns, 623 debug jobs, and three delegation studies, it reports major differences in tool-call formatting, session continuity, artifact validation, and verification across models and runtimes. The author explicitly says the studies were not pooled and do not rank models or runtimes.

    Hugging Face · Papers · 🔥 0
  22. Ollama v0.40.10 Adds NVIDIA Nemotron 3.5 Bolt E4B

    Ollama v0.40.10 adds support for NVIDIA’s Nemotron 3.5 Bolt E4B, a smaller vision-language model based on Gemma 4 and available through Ollama. According to the release notes, the model supports local conversations, image understanding, tool use, a 128K-token context window, and DFlash draft-based speculative decoding. The release also updates Gemma renderers and adds MLX support for Gemma 4 targets.

    Ollama Releases · 🔥 0
  23. Kapitsa 0.2 brings KFlash speculative decoding to Apple Silicon, tops self-run benchmark

    Kapitsa, an open-source inference engine for Apple Silicon aimed at Qwen and Nemotron models, released version 0.2 with a new speculative decoding method its developer calls KFlash, announced in a r/LocalLLaMA post. KFlash checks a best-first tree of the draft model's candidate tokens in one pass, whereas the comparison DFlash2 method checks a single chain and stops at the first wrong guess. In the developer's own benchmarks on a MacBook Pro M4 Pro with 48GB, Kapitsa reported 63 tokens/s on Qwen3.8 27B versus 53 tokens/s for Splash 1.4.0 with DFlash2, and 95 versus 62 tokens/s on Qwen3.5 4B on an M1 Pro 16GB machine; the author cautioned readers not to trust benchmarks and to run their own.

    r/LocalLLaMA · 🔥 0
  24. Hugging Face ships AnyLanguageModel 1.0 for Apple platforms

    Hugging Face has released AnyLanguageModel 1.0, alongside swift-huggingface 1.0 and swift-xet 1.0, a year after introducing the project. AnyLanguageModel is described as a drop-in replacement for Apple's Foundation Models framework that lets apps choose the model and provider, and the post says tool calling and guided generation now work with every provider, with image support using Foundation Models 27 syntax. Hosted support has expanded to OpenAI's Chat Completions and Responses APIs, Open Responses, Anthropic's Messages API with adaptive thinking and effort options, Google's Gemini API, Ollama, Private Cloud Compute on OS 27, and Hugging Face Inference Providers or other OpenAI-compatible endpoints via a custom base URL; the project also runs on Linux.

    Hugging Face · 🔥 0
  25. Kapitsa: open-source Apple Silicon inference engine tuned for Qwen and Nemotron

    Developer Ye at Nace.ai posted Kapitsa on r/LocalLLaMA, an Apache 2.0 open-source inference engine for Apple Silicon aimed at running coding agents on local models on Macs. Kapitsa supports only two model families — Qwen (3.5 4B, 3.8 27B, 3.6 35B-A3B and 3.8 Flash-Next) and NVIDIA Nemotron (3.5 Lightning, Bolt, Super-VL plus Dory 4B) — so the team can customize Metal kernels per architecture. The author published decode-speed tables on M1 Pro 16GB, M4 Pro 48GB and M5 Max 128GB machines, claiming it beats mlx-serve, oMLX, Ollama, MLX and llama.cpp in most cases, while trailing mlx-serve and oMLX by roughly 2-3% on Qwen3.8 Flash-Next. He also warns readers not to trust benchmarks, points them to a built-in kapitsa bench command to test themselves, notes that 4-bit quantization costs some quality, and says the new codebase should be expected to have bugs.

    r/LocalLLaMA · 🔥 0
  26. Ollama v0.40.7-rc0 adds MLX support for PPLX Decider v1.2

    Ollama tagged release v0.40.7-rc0 (commit 8be580a, tagged by dhiltgen on 17 Oct) with a title stating that the MLX component adds support for PPLX Decider v1.2 via PR #19072. The release description says the change implements a Qwen3.5-backed decision scorer with noncausal attention, a publisher readout, image handling and System One prompt encoding. Those capability details are stated only in the release text and have no independent verification.

    Ollama Releases · 🔥 0
  27. PrismML releases Ternary Bonsai 2 on-device models in 2B, 4B and 9B sizes

    PrismML released the Ternary Bonsai 2 series on October 15 in 2B, 4B and 9B sizes, aimed at running AI agents locally on phones, PCs and wearables for reasoning, image understanding and tool calling, IT之家 reported. The models are built on Alibaba's Qwen 3.5 base and use ternary weights encoded only as -1, 0 and +1; PrismML says they occupy roughly 0.43GB, 0.95GB and 2GB of storage. The company says it tested six capability areas across 19 benchmarks, with the 9B model using about 40% fewer tokens on standard instruction-following and everyday math tasks, and reports up to 5x throughput and 3x energy efficiency versus full-precision counterparts on RTX 4090 and M5 Pro, plus up to 34 tokens/s for the 9B and 120 tokens/s for the 2B on iPhone 18 Pro Max. Weights are released under Apache 2.0, runnable via llama.cpp on NVIDIA GPUs and MLX on Mac, iPhone and iPad, joining the existing Bonsai 2 27B.

    IT之家 AI · 🔥 0
  28. Soup CLI v0.76.0 refuses config settings that never took effect

    Soup, an open-source LLM fine-tuning CLI, shipped v0.76.0, which now refuses to load settings that were previously accepted but never actually applied — including training.quantization_aware: true, training.use_longlora and backend: unsloth on several tasks — so configs that loaded before may now fail, and the project advises running soup doctor --config soup.yaml first. The tool pitches one-command, single-YAML fine-tuning and an opt-in, still-BETA layer-streaming mode (stream_layers: true) that it says can LoRA fine-tune an 8B model on a 4 GB laptop GPU. The README also records a silent wrong-gradient defect on NF4 above roughly 165 MiB per layer that has been repaired, and notes that throughput figures from before and after the fix are not directly comparable.

    GitHub Trending · Python · 🔥 0
  29. Meta releases Muse Spark Open, an 816B open-weight multimodal model

    Meta released the weights of Muse Spark Open on 15 October, an open-weight mixture-of-experts model with about 816 billion total and 42 billion active parameters and a context window of up to one million tokens. It accepts text, images, audio and video as input and produces text, and Meta trained it for agentic coding, tool use and long-horizon tasks. Meta, which calls it the most capable US open-weight model to date, worked with Apple so the model can run locally on two M5 Ultra Mac Studio desktops via MLX; it is also offered through third-party APIs including Together AI, Baseten, Fireworks AI, Groq, Modal, Nebius and OpenRouter, with weights on Hugging Face under the Muse Model License. Arena.ai ranked it the top US open model on its Code Arena: WebDev leaderboard at 1511 points and 13th among open models overall, while an Artificial Analysis score of 39 cited in community discussion placed it behind GLM 5.3 and Kimi K3.

    Hugging Face · 🔥 0
  30. Apple MLX runs 816B Muse Spark Open locally on Mac Studio cluster

    Apple Machine Learning Research published a post on October 15, 2026, detailing how the latest MLX release runs the nearly trillion-parameter Muse Spark Open (816B total, 42B active) locally on Mac Studio. It says a single Mac Studio offers up to 512GB of unified memory and can scale across multiple Macs using an RDMA over Thunderbolt driver; on a cluster of four M5 Ultra Mac Studios, the model generated over 36 tokens per second at an 8,192-token context. Apple also compared M5 Ultra with M3 Ultra and benchmarked Mistral Medium 3.5, Qwen 3.8, and pipeline and tensor parallelization schemes, claiming up to 4.6x the start-up time-to-first-token speed of the previous Mac Studio generation.

    Apple Machine Learning Research · 🔥 0

Next →

Experience and discussion from the community

Share my MLX experienceAsk about MLX

jarvis: A private, local-first AI assistant with voice and text chat.

What problem it solves People often want an assistant that can participate in ongoing conversations, remember personal context, and use tools without requiring a cloud AI service. Jarvis addresses this with local-first voice and text interaction, personal memory, optional connected tools, and a setup process for locally controlled models. Key features Conversation-aware voice assistance — Jarvis m…

github.comhttps://github.com/isair/jarvis

TensorFold: A native language-model inference engine for serving checkpoints on Apple Silicon and NVIDIA GPUs.

What problem it solves Serving language models requires reading checkpoints, tokenizing requests, and running accelerated kernels while managing a local model server. TensorFold addresses these requirements with a native binary that can run on Apple Silicon and NVIDIA GPUs without requiring Python or MLX for serving. Key features Native Metal and CUDA inference — The Zig-based engine executes mode…

github.comhttps://github.com/ashhart/TensorFold

timesfm: A pretrained time-series foundation model for time-series forecasting.

What problem it solves Time-series forecasting requires models that can forecast univariate and multivariate series across different context lengths. Forecasters also need to incorporate optional covariates and produce quantile forecasts without requiring per-task tuning. Key features Univariate and multivariate forecasting — Forecast individual time series or multiple target variates from NumPy a…

github.comhttps://github.com/google-research/timesfm