RustingStrata

Headlines · 4

1. 16h
   Local LLM engines benchmarked on RTX 3060 with Qwen3.6-35B-A3B

   A Reddit user reported a single-system benchmark comparing RustingStrata, PolyStrata, and llama.cpp b12119 while running Qwen3.6-35B-A3B on an RTX 3060 12 GB. With a 5,290-token prompt, RustingStrata recorded 2,330 tok/s prefill versus 1,458–1,549 tok/s for PolyStrata and 1,692 tok/s for llama.cpp; no-MTP decode was reported at 90.9 tok/s, 78.5 tok/s, and 53.5 tok/s respectively. The comparison used different prompts and configurations, and the report notes that its PolyStrata no-MTP run still accepted some drafts.

   r/LocalLLaMA · 🔥 3
2. 1d
   RustingStrata Runs DeepSeek-V4.1-Flash on a Single RTX 3060

   A Reddit post reports that the pure-Rust GGUF inference engine RustingStrata can run the 249 GB DeepSeek-V4.1-Flash IQ2_XXS model on a single RTX 3060 12 GB with 30 GB RAM. It streams model experts from SSD through the page cache and uses a 9 GB LRU cache in VRAM, while attention and shared matrix-vector operations run on the GPU. In the submitter’s comparison, RustingStrata achieved 2–3.6 tok/s decoding versus 0.3–1.5 tok/s for llama.cpp b12116, but sampling and tool calls were not yet supported, the context cap was 4096, and results may vary with page-cache conditions.

   r/LocalLLaMA · 🔥 2
3. 5d
   RustingStrata vs. llama.cpp: Back-to-Back Qwen Benchmarks

   A Reddit user compared RustingStrata and llama.cpp on the same RTX 3060 12 GB system using the same Qwen GGUF files in back-to-back runs. Across models larger than VRAM, the user reports roughly 1.5× prefill and up to 2× decode performance for RustingStrata, while llama.cpp prefilled small dense models faster; the figures are means of two runs with desktop applications active and no tuning.

   r/LocalLLaMA · 🔥 0
4. 6d
   RustingStrata Brings Pure-Rust Qwen Inference to CUDA and Vulkan

   Developer Serious_Scarcity7818 introduced RustingStrata, a pure-Rust inference engine for Qwen GGUF models without ggml, supporting CUDA and Vulkan as well as hybrid CPU-GPU execution for MoE experts that exceed VRAM capacity. On the developer’s RTX 3060 12 GB system, it averaged about 36 tokens per second decoding Qwen3.8-Flash-Next, while a third-party tester reported roughly 209 tokens per second for Qwen3.6-35B-A3B on an RX 7900 XT.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my RustingStrata experienceAsk about RustingStrata