Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

tandem.cpp v0.3.0 improves prefill and adds Maya GGUF support

AI summary

tandem.cpp v0.3.0 is the project’s first tagged release. The developer reports that prompt processing is 2–6 times faster, and prompts run in 4096-token batches when the expert cache is enabled. The release also supports Project Maya’s DeepSeek-V4.1 Maya-XS GGUFs after a one-time conversion. Developer-run tests compared Qwen3.8 with llama.cpp, tandem.cpp and Strata, while a separate DeepSeek-V4.1 Maya-XS comparison used Project Maya. The release is currently Linux-only.

In-depth insight

The update targets local inference when a model is much larger than the combined VRAM and RAM, while adding a workflow for running DeepSeek-V4.1 Maya-XS GGUFs on tandem.cpp. The reported performance figures come from the developer’s own comparisons.

Thanks for the feedback on slow prompt processing. v0.3.0 is out (the first tagged release): https://github.com/ewhacc/tandem.cpp

Previous post: https://www.reddit.com/r/LocalLLaMA/comments/1xwan2j

tandem.cpp faster when the model is much bigger than VRAM + RAM together (like a 16 GB card with 32 GB of RAM), and runs various models that llama.cpp supports.

1. Prompts 2-6x faster

With the expert cache on, prompts now run in 4096-token batches. The bigger batch borrows VRAM only while a long prompt runs and gives it back afterwards. No flags needed.

RTX 5070 Ti 16 GB, 64 GB RAM, i5-12400F. The same prompts for every engine; Qwen3.8 with its MTP head in all three.

#tandem.cpp#llama.cpp#Project Maya#DeepSeek-V4.1#Qwen3.8#Strata#GGUF

Original · 2 min readRead the original
Group chat

No comments yet. Start the conversation.