User Reports Nearly 400 Tokens/sec for Qwen-Flash-Next
AI summary
A Reddit user says they built an inference engine that runs Qwen-Flash-Next at nearly 400 tokens per second on three 32GB NVIDIA V100 GPUs in an older Dell R740. The engine was originally forked from Strata, with its internals rebuilt for the older Volta generation. The performance claim has not been independently corroborated and the post provides limited technical detail.
In-depth insight
If reproducible, the reported setup would be relevant to efficient local inference of Qwen-Flash-Next on older NVIDIA hardware, though the evidence currently comes from a single user claim.