单张3090 Ti运行两款大模型
一名 r/LocalLLaMA 用户称，在配备单张 NVIDIA GeForce RTX 3090 Ti 和 128 GB 内存的系统上，DeepSeek-V4.1-Flash-1017 解码速度约为 35 tok/s，Qwen3.8-Flash-Next 则约为 138–140 tok/s。该帖称系统启用了 GPU Unlock，并配合自定义量化、内存管理和推测解码配置。这些数据来自单个用户的搭建，尚未得到独立验证。
这套可复现配置为单张消费级 GPU 运行大型混合专家模型提供了具体社区案例，但所报速度仍未得到独立确认。
I'm sharing this here because this is working way better than it used to a few months ago with older models. I see people here posting slower speeds with much better hardware than mine and hopefully this information helps.
I used gemini-flash to help me setup everything and I asked it to write up the sequence of steps to get me here - happy to provide more details if there's anything missing.
Since I use gentoo some tools/commands might be unfamiliar to you.
My 6 year old hardware:
- OS & CPU Gentoo Linux running on an AMD Ryzen Threadripper 3990X (64 cores, 128 threads).
- System Memory: 128 GB DDR4 RAM.
- GPU: 1 NVIDIA GeForce RTX 3090 Ti (24 GB GDDR6X VRAM, PCIe Gen 4 x16).
- Primary Storage: 4 TB SSD NVMe.
Models Used, Download Sources, and Measured Speeds
| Model | Architecture | Quantization / Files | Download Source | Real Measured Speed |
|---|---|---|---|---|
| DeepSeek-V4.1-Flash-1017 | 552B MoE (40 layers, 384 experts/layer, ~39B active) | XQ / GSQ V2 + rank-1017-085.bin + FP4 KV + Engram NVMe tables | IndrasMirror/DeepSeek-V4.1-Flash-1017-XQ-Uncensored | ~35 tok/s decode (with MTP/DSpark active) |
| Qwen3.8-Flash-Next | 125B MoE + 51B N-gram (GDN + QSA hybrid, ~3.5B active) | pack-qfn45 (4-bit intermediate, DeltaNet linear KV) | [IndrasMirror/Qwen3.8-Flash-Next-Swift-1.5-XQ-Uncensored](https://huggingface.co/IndrasMirror/Qwen3.8-Flash-Next-Swift-1.5-XQ-Uncensored) | ~138–140 tok/s decode |
#GPU Unlock#DeepSeek-V4.1-Flash-1017#Qwen3.8-Flash-Next#NVIDIA GeForce RTX 3090 Ti#AMD Ryzen Threadripper 3990X#Gemini Flash#Xolotl#DSpark