Community benchmark finds near-linear Qwen scaling to eight streams
A Reddit user reported spending about $40 on vast.ai instances to benchmark RAM-heavy inference servers with Qwen 3.8 Flash Next in Unsloth Q8. On systems with two AMD EPYC 7642 processors and 251GB or 504GB of DDR4 memory, the poster reported nearly linear per-stream scaling through eight concurrent streams at a 124k context window. Raw benchmark data was posted in the Reddit comments, and concurrency beyond eight streams was not tested.
The community test provides a practical data point for evaluating concurrent LLM inference on memory-heavy AMD server hardware, though its single-author setup and limited eight-stream ceiling constrain broader conclusions.
Raw data in the comments, if you go for that sort of thing.
The TL;DR
This build generates a nearly linear performance per stream up to at least 8 streams using a context window of 124k. I didn't test beyond that.
Abstract
Until today, I considered Strata a toy. I don't anymore.
Rented a few instances on vast.ai to do some benching. Blew about $40 in all on the benchmarking.
#Strata#Unsloth#Qwen 3.8 Flash Next#Maya#GLM 5.3F#DeepSeek V4.1F#AMD EPYC 7642#vast.ai