NVRTC
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- RustingStrata Brings Pure-Rust Qwen Inference to CUDA and Vulkan
Developer Serious_Scarcity7818 introduced RustingStrata, a pure-Rust inference engine for Qwen GGUF models without ggml, supporting CUDA and Vulkan as well as hybrid CPU-GPU execution for MoE experts that exceed VRAM capacity. On the developer’s RTX 3060 12 GB system, it averaged about 36 tokens per second decoding Qwen3.8-Flash-Next, while a third-party tester reported roughly 209 tokens per second for Qwen3.6-35B-A3B on an RX 7900 XT.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my NVRTC experienceAsk about NVRTC
Nobody has shared their experience with NVRTC yet.