llama.cpp PR proposes QSA selection without kq_mask
AI summary
A llama.cpp pull request by sanmai proposes building QSA selection without kq_mask for the qwen4exp branch. The report says the approach saves gigabytes of VRAM when running Qwen Flash Next and describes it as one way Strata improves use of the experts' cache.
In-depth insight
The proposal targets reducing VRAM use for Qwen Flash Next and is presented as improving the use of the experts' cache, according to the report.
| gigabytes of VRAM saved (Qwen Flash Next) "This is one of the things Strata does to squeeze the most juice from the experts' cache." submitted by /u/jacek2023 [link] [comments] |