Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

tandem.cpp

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 3

  1. tandem.cpp v0.3.0 improves prefill and adds Maya GGUF support

    tandem.cpp v0.3.0 is the project’s first tagged release. The developer reports that prompt processing is 2–6 times faster, and prompts run in 4096-token batches when the expert cache is enabled. The release also supports Project Maya’s DeepSeek-V4.1 Maya-XS GGUFs after a one-time conversion. Developer-run tests compared Qwen3.8 with llama.cpp, tandem.cpp and Strata, while a separate DeepSeek-V4.1 Maya-XS comparison used Project Maya. The release is currently Linux-only.

    r/LocalLLaMA · 🔥 0
  2. tandem.cpp adds AMD ROCm support and large-model CPU/GPU offloading

    The developer of tandem.cpp announced that its llama.cpp patch set can now build with HIP for AMD ROCm, while keeping frequently used MoE experts on the GPU and offloading remaining work to the CPU. In developer-run tests, DeepSeek-V4.1-Flash IQ2_XS with 139 GiB of experts reached 14.2 decode tokens per second on a system with an RTX 5070 Ti and 64 GB RAM, compared with 7.4 for llama.cpp. The AMD RX 7900 XTX test also included Qwen3.8 IQ3_S with MTP, but the reported performance figures have not been independently verified and Vulkan support is not yet available.

    r/LocalLLaMA · 🔥 0
  3. tandem.cpp accelerates CPU-assisted MoE decoding on consumer GPUs

    The independent tandem.cpp project has released patches for llama.cpp that keep frequently used MoE experts on the GPU while the CPU handles the rest, targeting single-user decoding when a model does not fit in VRAM. The project reports that Qwen3.8-Flash-Next reached 97.3 decoding tokens per second with greedy sampling on an i5-12400F system with 64 GB DDR4 memory and an RTX 4090, compared with 29.8 tokens per second for llama.cpp. It also reports gains for gpt-oss-120b and other MoE models, although the implementation is tuned specifically for Qwen3.8.

    r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my tandem.cpp experienceAsk about tandem.cpp

Nobody has shared their experience with tandem.cpp yet.