Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

omlx

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 5

  1. Three Used V100 GPUs Reach 100.8 Tokens/s With Qwen 3.6 35B

    A homelab builder measured 100.8 tokens per second for Qwen 3.6 35B at Q8 on three used Nvidia V100 GPUs with a 4K context, falling to 72.7 tokens per second at 128K. Across more than 130 llama.cpp tests, the dense Qwen 3.6 27B managed 24.3 tokens per second at 4K. In the reported 128K run, the three-V100 build consumed about half the GPU energy of the three-P100 setup. The listed $1,042 price covers only the GPUs and selected components, excluding RAM, storage, the case, cooling, and one of the two CPUs.

    Tom’s Hardware · 🔥 0
  2. Reddit user reports DeepSeek-V4.1-Flash speeds on Mac Studio M5 Ultra

    A Reddit user in r/LocalLLaMA reports running DeepSeek-V4.1-Flash-1017-oQ4e-mtp with omlx on a 512GB Mac Studio M5 Ultra. They report up to 1,100 tokens per second for prompt processing and 54 tokens per second for generation at a 128K context window. The user also says performance declines beyond 128K and concurrent requests divide the available generation throughput.

    r/LocalLLaMA · 🔥 0
  3. MiniMax highlights local quantization work for M3.1 Flash

    MiniMax highlighted community efforts to run its 428B-parameter MiniMax M3.1 Flash model locally using lower-bit weights, specialized kernels, and expert placement across GPUs and system RAM. The projects include Unsloth's 143.9 GB 2-bit Dynamic GGUF weights, ViC305's SAGE-EXL3 recipe for one DGX Spark, PeasantSmith's Project Maya quantizations, and additional work from MiaAI Lab, oMLX, and others. A LocalLLaMA review noted that the reported sizes, fidelity scores, and speed figures used separate evaluation sets and are not directly comparable.

    MiniMax_AI (X) · 🔥 0
  4. Reddit user reports GLM and Qwen results on a dual-M5U cluster

    A Reddit user reported running GLM 5.3 q6 and Qwen Flash Next q6 on two 256GB M5U units with 80 cores each. The post claims 4.9k prefill and 50 tokens per second for GLM, while Qwen reached 8k prefill and 200 tokens per second through Tensorfold; the author explicitly questioned the GLM prefill figure because it came from an LLM's explanation. The author said DeepSeek configured oMLX from guides and asked the community for optimization advice.

    r/LocalLLaMA · 🔥 0
  5. Kapitsa: open-source Apple Silicon inference engine tuned for Qwen and Nemotron

    Developer Ye at Nace.ai posted Kapitsa on r/LocalLLaMA, an Apache 2.0 open-source inference engine for Apple Silicon aimed at running coding agents on local models on Macs. Kapitsa supports only two model families — Qwen (3.5 4B, 3.8 27B, 3.6 35B-A3B and 3.8 Flash-Next) and NVIDIA Nemotron (3.5 Lightning, Bolt, Super-VL plus Dory 4B) — so the team can customize Metal kernels per architecture. The author published decode-speed tables on M1 Pro 16GB, M4 Pro 48GB and M5 Max 128GB machines, claiming it beats mlx-serve, oMLX, Ollama, MLX and llama.cpp in most cases, while trailing mlx-serve and oMLX by roughly 2-3% on Qwen3.8 Flash-Next. He also warns readers not to trust benchmarks, points them to a built-in kapitsa bench command to test themselves, notes that 4-bit quantization costs some quality, and says the new codebase should be expected to have bugs.

    r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my omlx experienceAsk about omlx

jarvis: A private, local-first AI assistant with voice and text chat.

What problem it solves People often want an assistant that can participate in ongoing conversations, remember personal context, and use tools without requiring a cloud AI service. Jarvis addresses this with local-first voice and text interaction, personal memory, optional connected tools, and a setup process for locally controlled models. Key features Conversation-aware voice assistance — Jarvis m…

github.comhttps://github.com/isair/jarvis