tandem.cpp v0.3.0提升预填充速度并支持Maya GGUF
tandem.cpp v0.3.0是该项目首个带标签的版本。开发者称提示处理速度提升2至6倍;启用专家缓存后,提示可按4096-token批次运行。此次更新还支持经过一次性转换的Project Maya DeepSeek-V4.1 Maya-XS GGUF。开发者自测的Qwen3.8对比涉及llama.cpp、tandem.cpp和Strata,Project Maya的对比则使用DeepSeek-V4.1 Maya-XS。该版本目前仅支持Linux。
此次更新面向模型容量远大于显存与内存总和的本地推理场景,并新增了在tandem.cpp上运行DeepSeek-V4.1 Maya-XS GGUF的流程。性能数据来自开发者自行进行的对比。
Thanks for the feedback on slow prompt processing. v0.3.0 is out (the first tagged release): https://github.com/ewhacc/tandem.cpp
Previous post: https://www.reddit.com/r/LocalLLaMA/comments/1xwan2j
tandem.cpp faster when the model is much bigger than VRAM + RAM together (like a 16 GB card with 32 GB of RAM), and runs various models that llama.cpp supports.
1. Prompts 2-6x faster
With the expert cache on, prompts now run in 4096-token batches. The bigger batch borrows VRAM only while a long prompt runs and gives it back afterwards. No flags needed.
RTX 5070 Ti 16 GB, 64 GB RAM, i5-12400F. The same prompts for every engine; Qwen3.8 with its MTP head in all three.
#tandem.cpp#llama.cpp#Project Maya#DeepSeek-V4.1#Qwen3.8#Strata#GGUF