Strata
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 73
- Transformers streams SSD-hosted experts for 180B MoE inference
Tarek Ziadé reported a Transformers implementation that streams quantized MoE experts from SSD into a bounded device cache, enabling the 180B-parameter Qwen3.8-Flash-Next model to run on a MacBook M5 with 32 GB of unified memory. The author measured 13.9 tokens per second after optimization and said this remained about 6% behind Strata, with per-layer synchronization identified as the remaining gap. The implementation is available through an open pull request, but its API was still under discussion with maintainers.
Hugging Face · 🔥 1 - Reddit user reports faster output speeds with Basalt than Strata
A Reddit user reports that Basalt produced 110–170 output tokens per second, compared with 70–110 tokens per second from Strata, using the same Qwen3.8 FlashNext-ud-q4_k_xl model. The user also says Basalt was less error-prone and that its reasoning was more on point. They describe Strata as a major improvement over llama.cpp, but say it sometimes confused its context after compaction and overthought tasks.
r/LocalLLaMA · 🔥 0 - Reddit commenter praises Strata’s hardware-aware local model approach
A Reddit commenter praises Strata as free and says its developer uses knowledge of computer hardware to run large models on average consumer hardware. The commenter also criticizes what they describe as sustained hostility toward Strata and comparable projects such as Ornith. The supplied report contains no benchmarks, technical documentation or developer statement verifying the capability claims, nor representative evidence for the broader sentiment assertions.
r/LocalLLaMA · 🔥 0 - Reddit User Reports Qwen Unsloth Q8 Outperforms Q3 in Financial PDF Test
A Reddit user described a self-hosted test of four models processing a large set of bank, credit-card, mortgage, brokerage, and retirement PDF statements. The user said Qwen FN Unsloth Q8 extracted all data accurately and produced a polished, insightful frontend, but ran for about eight hours. Qwen FN Q3 produced a nice frontend while extracting PDF data incorrectly and providing basic analytics. Orca nano1 finished in about two hours with mostly correct data but some clear errors and a basic frontend. Maya-S24 reportedly stalled in PDF parsing and validation, and the user stopped waiting after more than 10 hours.
r/LocalLLaMA · 🔥 0 - tandem.cpp v0.3.0 improves prefill and adds Maya GGUF support
tandem.cpp v0.3.0 is the project’s first tagged release. The developer reports that prompt processing is 2–6 times faster, and prompts run in 4096-token batches when the expert cache is enabled. The release also supports Project Maya’s DeepSeek-V4.1 Maya-XS GGUFs after a one-time conversion. Developer-run tests compared Qwen3.8 with llama.cpp, tandem.cpp and Strata, while a separate DeepSeek-V4.1 Maya-XS comparison used Project Maya. The release is currently Linux-only.
r/LocalLLaMA · 🔥 0 - A small LoRA can turn refusal ablations into a runtime option
A LocalLLaMA author says rank-1 refusal ablations from an abliterated Qwen3.8-Flash-Next model can be packaged as a small F32 GGUF LoRA and applied to an existing quantized model through llama.cpp. The post provides commands for loading the adapter or enabling it per request after a refusal, and says the technique also works in Strata and ds4/dwarfstar. The author reports Aider Polyglot, KLD, perplexity, and Jailbreak Bench results, but these are personal tests rather than independently validated findings.
r/LocalLLaMA · 🔥 0 - BigMoeOnEdge benchmarks MoE GGUF inference on local devices
The BigMoeOnEdge developer says the local inference tool accepts any MoE GGUF model loadable by llama.cpp without repacking or a special format. It supports NVIDIA, AMD, Intel, Mac, phone and CPU hardware across Windows, Linux, macOS, Android and iOS, with an --auto option for configuring a machine. The developer's benchmarks report strong results on 8–16 GB devices, but Strata was 1.3 times faster on two RTX 5060 Ti cards running Qwen3.8-Flash-Next.
r/LocalLLaMA · 🔥 0 - 1Cat-vLLM outpaces Strata on most Tesla V100 tests
A community benchmark on four 16 GB Tesla V100 GPUs found that 1Cat-vLLM delivered higher single-stream decode rates than Strata across all tested prompt lengths, including 124.9 versus 74.8 tokens per second at 2,654 tokens. It also led most prefill and concurrent-request tests, but Strata performed better on prefill near 100K tokens and generally achieved higher multi-token-prediction acceptance. The author says 1Cat-vLLM is a Tesla V100-focused vLLM fork with original expert-caching work.
r/LocalLLaMA · 🔥 0 - Reddit user reports Strata update degrades Qwen3.8-Flash-Next output
A Reddit user reports that updating Strata from version 0.1.42.0 made Qwen3.8-Flash-Next faster to start but less capable, with longer runtimes and more failures on a detailed voxel-art prompt. The user says version 0.1.42.0 completed similar work in roughly 15–30 minutes, or 50–60 minutes with xhigh, while the latest version often ran for at least 90 minutes, exhausted the 262K context, or failed. The report is a single user account and does not establish that the regression is reproducible across setups.
r/LocalLLaMA · 🔥 0 - tandem.cpp adds AMD ROCm support and large-model CPU/GPU offloading
The developer of tandem.cpp announced that its llama.cpp patch set can now build with HIP for AMD ROCm, while keeping frequently used MoE experts on the GPU and offloading remaining work to the CPU. In developer-run tests, DeepSeek-V4.1-Flash IQ2_XS with 139 GiB of experts reached 14.2 decode tokens per second on a system with an RTX 5070 Ti and 64 GB RAM, compared with 7.4 for llama.cpp. The AMD RX 7900 XTX test also included Qwen3.8 IQ3_S with MTP, but the reported performance figures have not been independently verified and Vulkan support is not yet available.
r/LocalLLaMA · 🔥 0 - Local test favors Qwen 3.8 Flash-Next over DeepSeek v4.1 Flash
A Reddit user in r/LocalLLaMA said they locally installed and benchmarked DeepSeek v4.1 Flash while waiting for Qwen 4. For their own workload, they found DeepSeek no better and substantially slower, while Strata running Qwen 3.8 Flash-Next offered the best value on their machine.
r/LocalLLaMA · 🔥 0 - Strata EXL3 AMD fork boosts decode speed and adds MiMo support
Developer nasone32 released an update to a Strata EXL3 fork for AMD, with improved HIP kernels and memory management producing a reported 10–20% average decode-speed increase, with some cases approaching 24%. The update also adds optional CPU-side expert repacking and experimental MiMo V2.6 Flash support, which the developer says ran at about 25–30 tokens per second on two Radeon RX 7900 XTX cards. Benchmarks were performed on RDNA3 hardware under Linux and ROCm, and the fork remains developed and tested primarily on that platform.
r/LocalLLaMA · 🔥 0 - Reddit user benchmarks MoE offloading on a 16GB RDNA4 9070 XT
A Reddit user compared MoE expert offloading across ExLlamaV3, Strata and llama.cpp on a 16GB RDNA4 9070 XT with 32GB DDR4, targeting 128K context with Q8 KV caching. The reported winners were ExLlamaV3 with DFlash2 for Ornith 1.5 35B-A3B and Qwen3.6-35B-A3B, and Strata with speculative decoding for Qwen3.8-Flash-Next Coder, with 44.0–71.6 tokens per second at 128K. The author said 16GB of VRAM constrains context length and quantization, and cautioned that the benchmark may be flawed.
r/LocalLLaMA · 🔥 0 - XeMaya brings native Intel Arc inference to recent MoE models
A developer built XeMaya, a fork of XeStrata that runs models natively on Intel Arc GPUs through SYCL and Level Zero. The early-alpha project reportedly supports DeepSeek V4.1 Flash and Coder, GLM 5.3 Flash, and Qwen3.8 Flash-Next, with preliminary prefill and generation results reported on an Ubuntu server. The developer cautioned that different model files, prompts, and GPU configurations made the comparisons uncontrolled, leaving further fixes and optimization needed.
r/LocalLLaMA · 🔥 0 - Reddit user reports Basalt outperforming Strata in local AI tests
A Reddit user compared Strata CLI engine 0.1.59 with Basalt 0.3.2 on Windows using Qwen3.8-Flash-Next IQ3S, a 256k context, and vision enabled. The user reported average generation speeds of 99.2 tokens per second for Strata and 100–118 tokens per second for Basalt, while Basalt also used less RAM and produced smoother, more reliable task completion. Basalt's peak prefill speed was reported as 4,200, although the user's average figure was uncertain.
r/LocalLLaMA · 🔥 0 - Reddit post claims Strata performance jump after optimizations
A Reddit user said Strata showed a performance jump after “heavy optimizations” and linked to the strata-5090-4070 repository. The post says the optimizations used both Opus 5.5 and Fable 5.2, and that the user’s 4070 Ti Super was connected through PCIe 3 x1. It does not quantify the improvement or provide benchmark figures, test conditions, or independent verification.
r/LocalLLaMA · 🔥 0 - Developer runs 775 GB Kimi-K3.1-Pro-Coder with three-tier offloading
Reddit developer Grawnch published a 775 GB quantization of Kimi-K3.1-Pro-Coder, down 56% from the 1.77 TB base configuration, together with the Strata loader and dr-triton dequantization kernels. Grawnch reports that the experimental setup ran all 93 layers on one RTX PRO 6000 Blackwell with system RAM and NVMe, reaching 36.8 batch tokens per second and 10.8 single-stream tokens per second while scoring 96.7% on AIME 2025 and LiveCodeBench.
r/LocalLLaMA · 🔥 0 - Reddit user creates Shinkansen motion graphics with local Qwen model
Reddit user /u/Yhojm reported creating a 15-second, 1920×1080, 60fps Shinkansen motion-graphics video with Qwen3.8 Flash Next GSQ RCO IQ3_XXS. The workflow used Strata and OpenCode to direct Three.js, Playwright and FFmpeg. The user said the first render took about 40 minutes and that roughly three follow-up prompts brought the total time to about two hours. On an RTX 4080 with 16GB VRAM, they reported about 95 tokens per second decoding and 3,000 tokens per second prefill. The first render was reportedly dark, lacked buildings and contained too much text.
r/LocalLLaMA · 🔥 0 - Betelgeuse developer reports faster out-of-core DeepSeek inference
A Betelgeuse developer reports running DeepSeek V4.1 Flash Q8_XL (1017) on an RTX 3060 12GB with 16GB RAM, achieving 61.80 tok/s warm prefill and roughly 18–19 tok/s decode. The implementation combines resident cache partitioning, single-pass layer-major prefill and end-of-pass VRAM consolidation to reduce the bottleneck in out-of-core mixture-of-experts inference. These results are self-reported, and the comparison with Strata uses different hardware configurations.
r/LocalLLaMA · 🔥 0 - Qwen Model Runs on Valve Steam Machine at About 18 Tokens per Second
A Reddit user running Qwen3.8-Flash-Next Gyro S on a Valve Steam Machine with an 8 GB AMD Navi 33 GPU reported about 18 tokens per second decoding and 235 tokens per second prefilling in Strata. Setting the resident budget to 64 GB improved decoding by roughly 20%, while a manually loaded MTP draft layer produced no gain and reduced prefilling by about 10%. The figures came from three runs per configuration on one system and were not independently verified.
r/LocalLLaMA · 🔥 0 - Reddit user runs Qwen Flash Next at 24 tokens per second on Strata
A Reddit user reports running Qwen Flash Next locally through Strata at 24 tokens per second on a 64GB M1 Max MacBook Pro. They say the model performs well for basic tasks but does not compete with larger frontier models. The report also mentions MoEShift and offloading larger models to SSD storage.
r/LocalLLaMA · 🔥 0 - Reddit user tests Unsloth Qwen Flash-Next quants at 12 GB VRAM
A r/LocalLLaMA post says Unsloth's new Dynamic v3.1 quantizations for Qwen Flash-Next and Qwen-27B, alongside ISTA-DASLab's advanced quantizations, make the models more accessible while keeping quality high. The author reports testing Unsloth's Iq2_XS quantization for Qwen Flash-Next with the Strata engine, running on 12 GB VRAM and 32 GB RAM at 1K prefill and about 50 tokens per second average decode across a 131K context window, with minimal quality loss under Strata's default auto-tuner. The post also says configurations targeting 8 GB VRAM with 32 GB system RAM are now in view, and that Qwen 4 was expected the same week.
r/LocalLLaMA · 🔥 0 - Laptop owner reports strong local LLM results with Strata
A r/LocalLLaMA user reports that Strata enabled an MSI Vector 16 with an Intel Core Ultra 9, RTX 5080 and upgraded laptop memory to run Qwen Q4 XL Flash at 48.9 tokens per second. The post also lists a combined decode/prefill figure of 1,860 tokens per second with a 50,000-token prompt, plus a reported 25 tokens per second for GLM 5.3 Flash in Q3_XSS. The owner says the roughly £3,100 setup can also use external GPUs for 80 GB of total VRAM.
r/LocalLLaMA · 🔥 0 - Qwen-Flash-Next benchmarked on a 16GB M2 Pro Mac Mini with Strata
A user loaded Strata and ran the iQ3_xxs quantization of Qwen-Flash-Next on a 16GB M2 Pro Mac mini, setting a maximum context of 128k, and used opencode on a separate workstation to build a script benchmarking performance and output quality on typical tasks such as large CSV sorting and low-end code work at varying context levels. The author reported it worked far better than expected. The author assumes speeds were always at their worst because nothing could be cached and experts were repeatedly re-read and swapped, which would mean prefill and decode token speed stay the same regardless of load. Median speed across tests from 16k to 120k context was 77 tok/s prefill and 5.5 tok/s decode. The author judged it barely passable as a chat generator but not hopeless as a last-resort "living on dial-up" code helper, and said it appears pretty rough on the SSD, though the write load is not much worse than using a Mini as an active DB server or other office server task.
r/LocalLLaMA · 🔥 0 - Steam Tags Explorer turns live Steam data into game ideas
A developer released Steam Tags Explorer, an open-source browser tool that uses current Steam tag data to help generate game-design ideas. It runs statically in the browser without external dependencies, refreshes its catalog daily, and includes gamified modes such as random crate unboxing. The developer built it with Qwen3.8 FN, Strata, and DeepSeek Harness in a fully local setup.
r/LocalLLaMA · 🔥 0 - Reddit user reports DeepSeek fixed HDMI display issue
A Reddit user said DeepSeek-V4.1-Flash-1017 resolved an HDMI display problem in about 15 minutes using the Strata inference engine. The same user reported that Qwen3.8-Flash-Next failed twice and took longer to finish, after testing on 128GB of DDR4 RAM and four RTX 3080 GPUs with 20GB VRAM each.
r/LocalLLaMA · 🔥 0 - Delta Inference Engine v0.93 expands local DeepSeek inference support
The developer released Delta Inference Engine v0.93 with support for 64, 96 and 128 GB of RAM, Unsloth quantization formats, EXL3 and basic multi-GPU configurations. On one Windows system with an RTX 3090, the reported Unsloth UD-IQ2_XXS configuration reached 41 tokens per second with 192 GB RAM and scored 57 out of 99 on the developer’s benchmark, compared with 52 for the previous default quantization and 59 for the full model via API. The update also includes public CI builds, faster prompt handling and fixes for long contexts and large model files.
r/LocalLLaMA · 🔥 0 - Reddit User Reports Local Agentic AI Stack on RTX 3060
A Reddit user in r/LocalLLaMA reported running Strata, Hermes, Kitten TTS with the Rosie voice, Whisper, and Qwen3.8-Flash-Next Q2_0 on an RTX 3060 setup with 48 GB of DRAM. The user said the system's capabilities exceeded their expectations and that talking to it felt like magic. They asked for recommendations for lightweight TTS and other models they could run.
r/LocalLLaMA · 🔥 0 - Qwen3.8 Flash Next Quant Report Nears BF16 Performance
A Reddit user reported that Unsloth’s benchmark chart placed the IQ_3S GSQ RCO quantization of Qwen3.8 Flash Next at 99.0% of the BF16 baseline across 12 benchmarks. The report also cited Unsloth’s claim that its UD v3.1 quantization is about 15% better than GSQ RCO at roughly the same file size, although that comparison is a vendor claim and the underlying chart is not independently verified here.
r/LocalLLaMA · 🔥 0 - RustingStrata Brings Pure-Rust Qwen Inference to CUDA and Vulkan
Developer Serious_Scarcity7818 introduced RustingStrata, a pure-Rust inference engine for Qwen GGUF models without ggml, supporting CUDA and Vulkan as well as hybrid CPU-GPU execution for MoE experts that exceed VRAM capacity. On the developer’s RTX 3060 12 GB system, it averaged about 36 tokens per second decoding Qwen3.8-Flash-Next, while a third-party tester reported roughly 209 tokens per second for Qwen3.6-35B-A3B on an RX 7900 XT.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my Strata experienceAsk about Strata
Nobody has shared their experience with Strata yet.