EPYC 7443 升级提升单台系统的本地 MoE 推理速度
一份 r/LocalLLaMA 用户实测报告称，该用户将 Core Ultra 7 265KF 和 MSI Z890 更换为 EPYC 7443 与 Supermicro H12SSL-NT，同时保留原有 GPU 配置。升级还将内存从 64 GB DDR5 扩充至 256 GB DDR4 ECC RDIMM，并拓宽了 PCIe 链路。调整 BMC 风扇阈值、内存超频、IOMMU 和 BIOS 设置后，作者报告 GLM-5.3-Flash 与 DeepSeek-V4.1-Flash 的解码速度提高约四倍，而 Qwen3.8-27B 密集模型的性能基本不变。作者认为，大幅 MoE 增益主要来自模型得以装入显存和内存，减少了每个词元从 NVMe 读取的数据量。这些结果仅来自一名用户的系统。升级后的 MoE 测试少用了一张 GPU，因此使用三张而非四张；测试期间 Maya 也从 1.0.34 或 1.1.20 更新至 1.1.42。
Same GPUs before and after:
- RTX 3090 FE - RTX 5070 Ti - 2× RTX 5060 Ti 16 GB (one is out waiting for a riser, so the big MoE runs below use 3 cards, against 4 before).
| Before | After | |
|---|---|---|
| CPU / board | Core Ultra 7 265KF, MSI Z890 | EPYC 7443 (24c, Zen 3, 4 CCDs), Supermicro H12SSL-NT |
| RAM | 64 GB DDR5-6000, 2 channels | 256 GB DDR4 ECC RDIMM, 8 channels (2666 sticks, run at 3200) |
| GPU links | x8 / x8 / x4 / x4 (chipset) | x16 / x16 / x8 |
| PSU | 1050 W | NZXT C1500 |
## What I had to fix or tune
- **Fans surging every few seconds.** The BMC's default "lower critical" fan threshold (420 RPM) was above what quiet fans idle at, so it kept declaring them dead and ramping everything to 100%. Fix: `ipmitool sensor thresh FANx lower 100 200 300` for each connected fan, then `ipmitool mc reset cold`. The BMC rounds to 140 RPM steps.
- **Over clocked RAM from 2666 to 3200.** Triad 88.6 → 102 GB/s; GLM decode increased about +3–5%. Stability: memtester (as root: as a normal user the memlock limit quietly shrinks it to ~8 MB per copy) plus a 3-hour DeepSeek run, confirming ECC (error corrections) at 0 corrected / 0 uncorrected throughout.
- **`iommu=pt` on the kernel command line.** +3.5% prefill for tensor-parallel llama.cpp, and NVMe random reads recovered (KIOXIA QD32 837K → 941K IOPS).
- **BIOS:** - APBDIS = 1 with fixed SOC P-state P0, which keeps the Infinity Fabric at full clock (about +20 W at idle). - Determinism slider set to Power. - **Resizable BAR on** (it was off: BAR1 was 256 MiB). - PCIe Ten Bit Tag on. Together: another +4% tensor-parallel prefill and a few % on GLM decode.
- **llama.cpp `-sm tensor`: `-ub 1024` instead of 512.** Without P2P, the two cards relay every layer's results through system RAM, across the I/O die. A bigger micro-batch halves the number of relays: +5% on Qwen3.8-27B, +9% on a 27B fine-tune. It needed a little more VRAM, so the split moved slightly toward the 3090.
#AMD EPYC 7443#Supermicro H12SSL-NT#Intel Core Ultra 7 265KF#MSI Z890#NVIDIA RTX 3090#NVIDIA RTX 5070 Ti#NVIDIA RTX 5060 Ti#llama.cpp#GLM-5.3-Flash#DeepSeek-V4.1-Flash#Qwen3.8-27B#Strata