RustingStrata推出纯Rust Qwen推理引擎
开发者Serious_Scarcity7818推出RustingStrata，这是一款不依赖ggml、面向Qwen GGUF模型的纯Rust推理引擎，支持CUDA、Viulkan，以及通过CPU-GPU混合执行运行超出显存容量的MoE专家。开发者称，在RTX 3060 12 GB设备上运行Qwen3.8-Flash-Next时，平均解码速度约为36 tokens/s；第三方测试者则称RX 7900 XT运行Qwen3.6-35B-A3B约为209 tokens/s。
RustingStrata为本地运行Qwen模型提供了新的纯Rust路线，并通过CPU-GPU卸载支持大型MoE模型，但多数性能数据仍来自开发者自测。
I've been working on this for a while: RustingStrata. It's an inference engine for Qwen GGUF models, written in pure Rust, no ggml. The CUDA kernels get compiled at startup with NVRTC, and the same kernels also run on Vulkan.
It runs a lot of Qwen models by now:
- Qwen3.5 / 3.6: 2B up to 35B-A3B
- Qwen3: 4B dense, 30B-A3B
- Qwen3.8-27B dense
- Qwen3-Next-80B-A3B
#RustingStrata#Qwen#Qwen3.8-Flash-Next#Qwen3.6-35B-A3B#Qwen3-Next-80B-A3B#Strata#llama.cpp#CUDA#Vulkan#NVRTC#NVIDIA RTX 3060#AMD Radeon RX 7900 XT