Atomic.Chat发布修改版DeepSeek量化模型
AI 总结
Atomic.Chat发布了修改后的DeepSeek-V4.1-Flash-1017 GSQ-RCO量化分片,仅调整每个文件的82个密集张量,并保留原路由专家、文件头、文件大小及Engram文件兼容性。团队称,三种量化版本在JailbreakBench上的拒答率由85.2%至88.9%降至0%,但部分思考过程中仍会拒答。测试通过Strata Engine在8K上下文和租用GPU上完成,尚未验证长上下文性能及12 GB GPU上的速度。
深度见解
该版本为降低高量化DeepSeek模型的拒答率提供了具体方法,同时保留原有GSQ-RCO路由专家,但数据来自团队自测,硬件验证范围有限。
| Measured, refusals 85%→0% even in thinking, 3 sizes 12 GB GPU VRAM, 64GB RAM + SSD via Strata Engine We took ISTA-DASLab's GSQ-RCO quants of DeepSeek-V4.1-Flash-1017 (the ones people run in Strata with a 12 GB GPU and the Engram table on SSD) and removed the refusals. We only touched 82 dense tensors per file: attention output, shared expert down proj and the Engram value proj. The 384 routed experts are byte for byte the same as ISTA's, so their GSQ rounding is kept. Headers and file sizes match too, so you just swap the shards and keep using ISTA's Engram file. Refusals are on JailbreakBench, stock rung -> ours, same Strata setup: We also ran GPQA-Diamond and AIME25 on IQ2_XXS against stock to make sure it didn't get dumber. GPQA came out 83.3% vs 83.3%, AIME 80.0% vs 76.7% (only 30 problems, so call it the same). Things to know: Model: https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF All the measurements: https://huggingface.co/datasets/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GGUF-metrics Strata Engine: https://github.com/niko1221/strata Thanks to DeepSeek for the base model (MIT), ISTA-DASLab for GSQ-RCO and the Engram file, and the Strata dev for the engine. I'm on the Atomic.Chat team that made this. submitted by /u/gladkos [link] [comments] |
#Atomic.Chat#DeepSeek-V4.1-Flash-1017#ISTA-DASLab#Strata Engine#JailbreakBench#GSQ-RCO#Engram