Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

strata

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 73

  1. Reddit user reports 70–90 tokens/s for a €1,600 local Qwen setup

    A Reddit user described a local inference setup built around an Intel Core i7-14700KF, 64 GB of RAM and a Radeon RX 9070 XT with 16 GB of memory. Using the Strata repository and a Qwen3.8-Flash-Next-GSQ-RCO-Q2_0 model, the user reported inference at 70–90 tokens per second and described the system as costing about €1,600.

    r/LocalLLaMA · 🔥 0
  2. llama.cpp PR proposes QSA selection without kq_mask

    A llama.cpp pull request by sanmai proposes building QSA selection without kq_mask for the qwen4exp branch. The report says the approach saves gigabytes of VRAM when running Qwen Flash Next and describes it as one way Strata improves use of the experts' cache.

    r/LocalLLaMA · 🔥 0
  3. Seven Qwen 3.8 FN Inference Engines Benchmarked on One RTX 5090

    The developer of Basalt benchmarked seven Qwen 3.8 FN inference engines on an RTX 5090 with 128 GB of RAM. The report’s results placed Basalt and Infernix ahead overall: Basalt recorded 354 tokens per second for structured output and 190 tokens per second at 25K context, while both reached a 3.94-second time to first token at 25K. Each engine converted NVIDIA’s NVFP4 weights with its own tool and was configured according to its documentation. Every engine received the same seven greedy requests, with one cold run per request and no warm-up.

    r/LocalLLaMA · 🔥 0
  4. Strata now supports Qwen 3.6 35B

    A Reddit post reports that Strata now supports Qwen 3.6 35B. In the same post, a user said a Strata build for the gfx906 architecture on a 16GB AMD MI50 performed worse than llama.cpp in decode speed. The report did not provide official confirmation or broader performance benchmarks.

    r/LocalLLaMA · 🔥 0
  5. Reddit author publishes Strata-ready Qwen 3.8 Flash Next IQ4_XS clone

    Reddit user jjusko20 updated a community distribution of an uncensored Swift 1.5 Qwen 3.8 Flash Next clone quantized with Unsloth Dynamic IQ4_XS. The author says it was quantized with a custom importance matrix for English, tool use, math, code and reasoning, and includes Strata artifacts such as an MTP head and vision files. The author reports a mean KLD of 0.06 and median of 0.02, while noting that the mean was concentrated on drug-related prompts and skewed in that direction. The uncensoring method, package contents and quality figures have not been independently verified.

    r/LocalLLaMA · 🔥 0
  6. capy-flash v0.7.1 runs 125B Qwen3.8-Flash-Next in 64 GB of RAM

    The developer of capy-flash, an open-source inference engine for running Qwen3.8-Flash-Next (125B MoE) and Qwen3.8-27B on consumer RTX 50 cards, released v0.7.1, which he says cuts Flash-Next's RAM requirement to 64 GB on one or two GPUs, down from the two 16 GB cards and 96 GB of RAM his setup needed before. The release uses ISTA-DASLab's ~3-bit GSQ-RCO i-quants for the routed experts (about 48 GiB in RAM instead of 65 for NVFP4), keeps hot experts in VRAM and computes the rest on the CPU, with peak RAM around 57 GB. It also fits Qwen3.8-27B with 3-bit weights on a single 16 GB card at 64k context with vision and MTP, adds AVX2 CPU support and guards against empty replies. The developer reports 234 tok/s for Flash-Next on two 16 GB cards and 174 tok/s for the 27B on one card, measured on an RTX 5070 Ti with a Ryzen 9 9900X, and notes 3-bit weights cost quality on the 27B: MMLU-Pro falls from 0.78 to about 0.72 while GSM8K is on par with NVFP4.

    r/LocalLLaMA · 🔥 0
  7. Reddit user praises Qwen 3.8 Flash Next running locally in strata at IQ4_XS

    A Reddit user on r/LocalLLaMA said they finally set up strata after holding off and came away "pretty blown away" by Qwen 3.8 Flash Next running in strata at IQ4 xs. They said a 27B model at Q6 kv8 had been their daily driver and worked well, but Flash Next in that configuration was "really winning me over." The user described the model as not perfect but comparable to models they were using professionally the previous year.

    r/LocalLLaMA · 🔥 0
  8. Strata 0.2.0 beta cuts local LLM hardware needs by up to 16GB

    AIbase AI新闻 reported on October 20 that Strata, an open-source local LLM runner, released its 0.2.0 beta with a new standalone GUI and a claimed minimum of about 27.6GB combined RAM and VRAM — for example, a 12GB-VRAM GPU with 20GB of system RAM. The report says recommended tiers drop from 48GB to 40GB for IQ2_XS and from 64GB to 48GB for IQ3_XXS, an 8–16GB reduction, with all figures coming from this single report. It also cites performance gains of up to about 13% for RTX 30-series text generation, near-doubled prompt processing on some models for AMD RX 9000 cards and about 42% for the Intel Arc A750, alongside Windows, macOS and Linux support and a native Chinese interface.

    AIbase AI新闻 · 🔥 0
  9. Berth v0.3 parks local AI engines in system RAM to speed GPU switching

    Berth, a system service its developer describes in a r/LocalLLaMA post, watches the GPU and swaps and loads whichever engine the user wants, so that VRAM is dedicated to each running function without manual swaps or leftover processes holding memory. The developer has released v0.3, which parks engines in system RAM instead of unloading them, so switching back to a parked engine skips the reload from disk. They are looking for testers with 64, 96 or 128 GB of RAM and a 24 GB card, multi-GPU machines, Windows with an NVIDIA card, and AMD on Linux, noting that their own 62 GB workstation cannot measure parking at larger sizes because Strata alone pins 40 GB.

    r/LocalLLaMA · 🔥 0
  10. Qwen3.8-27B and Qwen3.8-Flash-Next local inference benchmark roundup

    An r/LocalLLaMA user, FeydRowan, compiled single-request decode speeds for Qwen3.8-27B and the 125B MoE Qwen3.8-Flash-Next on consumer hardware, marking rows as “verified” only when backed by a LocalMaxxing run or a paired run on the same machine. The author's own capy-flash rows were measured on 2× RTX 5070 Ti with 96 GB DDR5, while most other figures are as reported by each project; the post also compares tensor parallelism versus layer splitting, expert offload, quant formats and drafters, and calculates tokens per second per €1,000 using EU VAT-inclusive prices. It puts 2× R9700 on top for Flash-Next cost-efficiency and says 96 GB of DDR5 is about 40% of the bill for such builds, but the performance and price data are largely user-compiled and self-reported.

    r/LocalLLaMA · 🔥 0
  11. Xolotl engine runs 552B DeepSeek-V4.1-Flash on a single RTX 4090, developer reports

    Reddit user indrasmirror released Xolotl, a Strata-based inference engine, claiming it runs the full 552B DeepSeek-V4.1-Flash on one gaming PC: a single RTX 4090 (24GB), a Ryzen 7 5700X, 90GB of DDR4 and an NVMe drive, with weights spread across VRAM, pinned RAM and NVMe and only the experts each token needs pulled in. The developer self-reports about 27 tok/s decode, 180–320 tok/s prefill, 512K context, and 21/21 HumanEval, 10/10 tool calls and 3/3 agentic bug-fix loops through the real server with thinking on; none of these figures has been independently verified. The post also says the same engine runs GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3.8-27B, that a 152 GiB uncensored V4.1 quant is on Hugging Face, and that the 510GB DeepSeek-V4.1-Flash-1017 update will be abliterated and requantized within a couple of days.

    r/LocalLLaMA · 🔥 0
  12. AI memory demand squeezes PC buyers as Q3 shipments fall 20%

    AI-server demand for HBM and server DRAM is crowding out consumer memory capacity: Omdia estimates memory and SSDs have climbed from roughly 15% of PC bill-of-materials cost to nearly 40%, while IDC preliminary data put Q3 2026 global PC shipments down 20.1% year over year (78.5M to 62.7M units) and Omdia measured a 21.2% decline in desktops, notebooks and workstations, with Macs also down. TrendForce calculates CPU, DRAM and SSD now account for about 68% of the BOM in a $900 mainstream laptop; Gigabyte responded on Oct. 9 by saying all its B760 and H610 boards, DDR4 included, will support new Intel LGA1700 processors expected in early 2027 via BIOS update, after adding AM4 boards alongside the Ryzen 7 5800X3D relaunch in August. On the software side, the open-source Strata project, with about 19,500 GitHub stars, reports running the 125B-parameter Qwen3.8-Flash-Next at roughly 94 tokens/s under Q2_0 quantization on a PC with an RTX 5070 (12GB VRAM), Ryzen 5 7600 and 64GB of system RAM, or about 53 tokens/s on IQ3_S.

    虎嗅 AI · 🔥 0
  13. Strata reportedly runs 125B Qwen model on 12GB GPUs

    Developer Niko1221 has open-sourced the Strata engine, which reportedly runs a quantized Qwen3.8-Flash-Next model with 125 billion parameters on consumer GPUs with at least 12GB of VRAM. The engine keeps the MoE model in system RAM, loads frequently used experts into VRAM, and uses a lightweight model for speculative decoding; reported tests reached 94 tokens per second on an RTX 5070 with a 2-bit quantization.

    IT之家 AI · 🔥 0

← Previous

Experience and discussion from the community

Share my strata experienceAsk about strata

Nobody has shared their experience with strata yet.