Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

ISTA releases 1-bit DeepSeek-V4.1-Flash-1017 GSQ-RCO Coder with pruned experts

AI summary

The ISTA Deep Algorithms and Systems Lab says it has released a Coder build of DeepSeek-V4.1-Flash-1017, a sparse mixture-of-experts model with 384 routed experts per layer (6 active per token) and 763.2B parameters, 551.9B of them in the transformer. The build combines GSQ (Gumbel-Softmax Quantization) with RCO (Riemannian Constrained Optimization) to prune half the routed experts, keeping 192 of 384 per layer and cutting transformer weights to 72.1 GB from 138.2 GB in the unpruned IQ2_XXS build, which the lab rounds to 1 bit per parameter. Reported results show LiveCodeBench v6 at 88.57 versus 89.71 for the base model and SWE-bench Verified at 77.00 versus 81.20, and the GGUF runs in llama.cpp b11870 or newer and in Strata.

In-depth insight

The release pairs low-bit quantization with budget-constrained expert pruning to cut a 763B MoE model's weight footprint roughly in half while, per the lab, retaining most of its coding benchmark scores, and it ships as downloadable GGUF files for llama.cpp and Strata.

We have released a Coder build of DeepSeek-V4.1-Flash-1017, compressed with GSQ and RCO, in which half of the model's routed experts have been removed. DeepSeek-V4.1-Flash-1017 is a sparse mixture-of-experts model: 384 routed experts per layer, of which 6 are active per token, 763.2B parameters, 551.9B of them in the transformer. What's inside Why one budget per layer GGUF stores a single expert count for the whole model, so every layer must keep exactly the same number of experts; uneven layers would need padding that gives back the memory pruning saved. RCO treats each layer's budget as a hard constraint and holds all of them exactly at every optimisation step, while the layers are still optimised jointly through one loss on the full model. A penalty formulation would only meet the budgets approximately and would need one tuned coefficient per layer. Results SWE-bench Verified runs long agentic trajectories over real repositories, and tolerates the capacity reduction less well than single-problem code generation, as expected when errors accumulate across turns. Real-world examples, each a single HTML file written by the Coder: an animated SVG of a pelican riding a bicycle, a three.js voxel pagoda garden, and a playable first-person voxel sandbox. For full post incl. videos see: https://x.com/mhelcig/status/2117257900863156518 Links The repository ships the complete per-tensor RCO allocation. The GGUF runs in llama.cpp b11870 or newer and in Strata. The Coder build is an experimental release and feedback is welcome, particularly on capabilities that were not represented in the calibration mixture. Requests for models to quantize or prune are also welcome. From the ISTA Deep Algorithms and Systems Lab. submitted by /u/Loginhe [link] [comments]

#DeepSeek#ISTA Deep Algorithms and Systems Lab#DeepSeek-V4.1-Flash-1017#GSQ#RCO#llama.cpp#Strata

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.