tandem.cpp v0.3.0 improves prefill and adds Maya GGUF support
tandem.cpp v0.3.0 is the project’s first tagged release. The developer reports that prompt processing is 2–6 times faster, and prompts run in 4096-token batches when the expert cache is enabled. The release also supports Project Maya’s DeepSeek-V4.1 Maya-XS GGUFs after a one-time conversion. Developer-run tests compared Qwen3.8 with llama.cpp, tandem.cpp and Strata, while a separate DeepSeek-V4.1 Maya-XS comparison used Project Maya. The release is currently Linux-only.
The update targets local inference when a model is much larger than the combined VRAM and RAM, while adding a workflow for running DeepSeek-V4.1 Maya-XS GGUFs on tandem.cpp. The reported performance figures come from the developer’s own comparisons.
Thanks for the feedback on slow prompt processing. v0.3.0 is out (the first tagged release): https://github.com/ewhacc/tandem.cpp
Previous post: https://www.reddit.com/r/LocalLLaMA/comments/1xwan2j
tandem.cpp faster when the model is much bigger than VRAM + RAM together (like a 16 GB card with 32 GB of RAM), and runs various models that llama.cpp supports.
1. Prompts 2-6x faster
With the expert cache on, prompts now run in 4096-token batches. The bigger batch borrows VRAM only while a long prompt runs and gives it back afterwards. No flags needed.
RTX 5070 Ti 16 GB, 64 GB RAM, i5-12400F. The same prompts for every engine; Qwen3.8 with its MTP head in all three.
#tandem.cpp#llama.cpp#Project Maya#DeepSeek-V4.1#Qwen3.8#Strata#GGUF