Qwen-Flash-Next
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 2
- Qwen-Flash-Next benchmarked on a 16GB M2 Pro Mac Mini with Strata
A user loaded Strata and ran the iQ3_xxs quantization of Qwen-Flash-Next on a 16GB M2 Pro Mac mini, setting a maximum context of 128k, and used opencode on a separate workstation to build a script benchmarking performance and output quality on typical tasks such as large CSV sorting and low-end code work at varying context levels. The author reported it worked far better than expected. The author assumes speeds were always at their worst because nothing could be cached and experts were repeatedly re-read and swapped, which would mean prefill and decode token speed stay the same regardless of load. Median speed across tests from 16k to 120k context was 77 tok/s prefill and 5.5 tok/s decode. The author judged it barely passable as a chat generator but not hopeless as a last-resort "living on dial-up" code helper, and said it appears pretty rough on the SSD, though the write load is not much worse than using a Mini as an active DB server or other office server task.
r/LocalLLaMA · 🔥 0 - User Reports Nearly 400 Tokens/sec for Qwen-Flash-Next
A Reddit user says they built an inference engine that runs Qwen-Flash-Next at nearly 400 tokens per second on three 32GB NVIDIA V100 GPUs in an older Dell R740. The engine was originally forked from Strata, with its internals rebuilt for the older Volta generation. The performance claim has not been independently corroborated and the post provides limited technical detail.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my Qwen-Flash-Next experienceAsk about Qwen-Flash-Next
Nobody has shared their experience with Qwen-Flash-Next yet.