ISTA-DASLab

Headlines · 5

1. 4d
   Developer Runs DeepSeek-V4.1-Flash-1017 Q2 at 41 Tokens per Second

   A community developer reported running the DeepSeek-V4.1-Flash-1017 Q2 GGUF natively on Windows with one RTX 3090 and 192 GB of RAM, without WSL. On that system, the developer measured an average of 41 tokens per second and next-token agreement with DeepSeek's API of 90.2% on general text and 92.4% on code and mathematics. The developer also claimed the quantization fits the full 1M-token context and said the interface engine supports several published quantization sets.

   r/LocalLLaMA · 🔥 0
2. 4d
   Community test compares 2.25bpw DeepSeek-V4.1-Flash with official API

   A r/LocalLLaMA author compared ISTA-DASLab's DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF 2.25bpw quantization with the official DeepSeek API on a 100-question, text-only subset of Humanities Last Exam. Among 74 questions the API completed, semantic accuracy was 19/74 for the local quantization and 21/74 for the API; the author called this a preliminary smoke test because differing token limits, non-completion, and substantial run-to-run variance limited the comparison.

   r/LocalLLaMA · 🔥 0
3. Oct 23
   ISTA-DASLab releases 1.75–2.25-bit DeepSeek-V4.1-Flash GGUFs

   ISTA-DASLab announced three GSQ-RCO GGUF builds of DeepSeek-V4.1-Flash-1017: IQ1_M at 1.75 bits per weight, IQ2_XXS at 2.00 bpw, and IQ2_XS at 2.25 bpw, with backbones ranging from 120.7GB to 155.2GB. The release post says IQ2_XXS matched the base model on the evaluated benchmarks at reasoning effort 50, while a community discussion questioned the saturated benchmarks and noted that no independent KLD or perplexity measurements were supplied. The models reportedly run through Strata on a single 12GB GPU but cannot yet run in llama.cpp because architecture support has not been merged.

   r/LocalLLaMA · 🔥 0
4. Oct 22
   capy-flash v0.7.1 runs 125B Qwen3.8-Flash-Next in 64 GB of RAM

   The developer of capy-flash, an open-source inference engine for running Qwen3.8-Flash-Next (125B MoE) and Qwen3.8-27B on consumer RTX 50 cards, released v0.7.1, which he says cuts Flash-Next's RAM requirement to 64 GB on one or two GPUs, down from the two 16 GB cards and 96 GB of RAM his setup needed before. The release uses ISTA-DASLab's ~3-bit GSQ-RCO i-quants for the routed experts (about 48 GiB in RAM instead of 65 for NVFP4), keeps hot experts in VRAM and computes the rest on the CPU, with peak RAM around 57 GB. It also fits Qwen3.8-27B with 3-bit weights on a single 16 GB card at 64k context with vision and MTP, adds AVX2 CPU support and guards against empty replies. The developer reports 234 tok/s for Flash-Next on two 16 GB cards and 174 tok/s for the 27B on one card, measured on an RTX 5070 Ti with a Ryzen 9 9900X, and notes 3-bit weights cost quality on the 27B: MMLU-Pro falls from 0.78 to about 0.72 while GSM8K is on par with NVFP4.

   r/LocalLLaMA · 🔥 0
5. Oct 20
   ISTA-DASLab prefiller recovers Qwen3.8-27B low-quant accuracy via SSD streaming

   A r/LocalLLaMA post describes an ISTA-DASLab demo that uses a prefiller model streamed from SSD to recover the accuracy lost by low-bit quantization of Qwen3.8-27B, while keeping VRAM use low and decoding fast. The post says the accompanying graph shows MMLU-Pro accuracy and points to a Hugging Face page, ISTA-DASLab/Qwen3.8-27B-NVFP4-prefiller. The poster notes the demo requires an Nvidia Blackwell GPU and was therefore not tested by them, and that the claim of no speed penalty versus running without the prefiller for contexts longer than about 4k is an untested expectation.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my ISTA-DASLab experienceAsk about ISTA-DASLab