Strata reportedly runs 125B Qwen model on 12GB GPUs
Developer Niko1221 has open-sourced the Strata engine, which reportedly runs a quantized Qwen3.8-Flash-Next model with 125 billion parameters on consumer GPUs with at least 12GB of VRAM. The engine keeps the MoE model in system RAM, loads frequently used experts into VRAM, and uses a lightweight model for speculative decoding; reported tests reached 94 tokens per second on an RTX 5070 with a 2-bit quantization.
It could lower the hardware barrier for running very large open models locally, although the reported performance comes from third-party coverage and specific configurations.
IT之家 10 月 6 日消息,科技媒体 gigazine 今天(10 月 6 日)报道,报道称开发者 Niko1221 开源推出 Strata 引擎,可以在 12GB 及以上显存的消费级显卡上,运行量化的 Qwen3.8-Flash-Next 模型(1250 亿参数)。

IT之家注:Qwen3.8-Flash-Next 模型是阿里巴巴 Qwen 团队于 2026 年 8 月推出的多模态混合专家(MoE)模型的压缩版本,配有 125B 参数,另含 51B 参数的 n-gram 嵌入表,原生支持 262K token 上下文长度。

Strata 为了降低显存占用,主要采用两项关键技术:其一是将 MoE(混合专家)模型整体载入 RAM,仅将高频使用的专家模型载入 VRAM。其二使用轻量模型进行投机解码,预测下一 Token 以加速推理过程。
#Strata#Niko1221#Qwen3.8-Flash-Next#Qwen#Alibaba#NVIDIA GeForce RTX 5070#AMD Radeon RX 9070 XT