tandem.cpp
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 3
- tandem.cpp v0.3.0 improves prefill and adds Maya GGUF support
tandem.cpp v0.3.0 is the project’s first tagged release. The developer reports that prompt processing is 2–6 times faster, and prompts run in 4096-token batches when the expert cache is enabled. The release also supports Project Maya’s DeepSeek-V4.1 Maya-XS GGUFs after a one-time conversion. Developer-run tests compared Qwen3.8 with llama.cpp, tandem.cpp and Strata, while a separate DeepSeek-V4.1 Maya-XS comparison used Project Maya. The release is currently Linux-only.
r/LocalLLaMA · 🔥 0 - tandem.cpp adds AMD ROCm support and large-model CPU/GPU offloading
The developer of tandem.cpp announced that its llama.cpp patch set can now build with HIP for AMD ROCm, while keeping frequently used MoE experts on the GPU and offloading remaining work to the CPU. In developer-run tests, DeepSeek-V4.1-Flash IQ2_XS with 139 GiB of experts reached 14.2 decode tokens per second on a system with an RTX 5070 Ti and 64 GB RAM, compared with 7.4 for llama.cpp. The AMD RX 7900 XTX test also included Qwen3.8 IQ3_S with MTP, but the reported performance figures have not been independently verified and Vulkan support is not yet available.
r/LocalLLaMA · 🔥 0 - tandem.cpp accelerates CPU-assisted MoE decoding on consumer GPUs
The independent tandem.cpp project has released patches for llama.cpp that keep frequently used MoE experts on the GPU while the CPU handles the rest, targeting single-user decoding when a model does not fit in VRAM. The project reports that Qwen3.8-Flash-Next reached 97.3 decoding tokens per second with greedy sampling on an i5-12400F system with 64 GB DDR4 memory and an RTX 4090, compared with 29.8 tokens per second for llama.cpp. It also reports gains for gpt-oss-120b and other MoE models, although the implementation is tuned specifically for Qwen3.8.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my tandem.cpp experienceAsk about tandem.cpp
Nobody has shared their experience with tandem.cpp yet.