EPYC 7443 upgrade boosts local MoE inference in one system
In a field report, an r/LocalLLaMA user replaced a Core Ultra 7 265KF and MSI Z890 with an EPYC 7443 and Supermicro H12SSL-NT while retaining the same GPU inventory. The upgrade also increased memory from 64 GB DDR5 to 256 GB DDR4 ECC RDIMM and widened PCIe links. After tuning BMC fan thresholds, RAM overclocking, IOMMU and BIOS settings, the author reported GLM-5.3-Flash and DeepSeek-V4.1-Flash decode speeds increasing by roughly fourfold, while Qwen3.8-27B dense-model performance was broadly unchanged. The author attributes the large MoE gains mainly to the models fitting in VRAM and RAM, with less data being read from NVMe storage. These are results from one user's system. One GPU was unavailable during the post-change MoE tests, so those runs used three GPUs rather than four, and Maya was updated from versions 1.0.34 or 1.1.20 to 1.1.42 during testing.
Same GPUs before and after:
- RTX 3090 FE - RTX 5070 Ti - 2× RTX 5060 Ti 16 GB (one is out waiting for a riser, so the big MoE runs below use 3 cards, against 4 before).
| Before | After | |
|---|---|---|
| CPU / board | Core Ultra 7 265KF, MSI Z890 | EPYC 7443 (24c, Zen 3, 4 CCDs), Supermicro H12SSL-NT |
| RAM | 64 GB DDR5-6000, 2 channels | 256 GB DDR4 ECC RDIMM, 8 channels (2666 sticks, run at 3200) |
| GPU links | x8 / x8 / x4 / x4 (chipset) | x16 / x16 / x8 |
| PSU | 1050 W | NZXT C1500 |
## What I had to fix or tune
- **Fans surging every few seconds.** The BMC's default "lower critical" fan threshold (420 RPM) was above what quiet fans idle at, so it kept declaring them dead and ramping everything to 100%. Fix: `ipmitool sensor thresh FANx lower 100 200 300` for each connected fan, then `ipmitool mc reset cold`. The BMC rounds to 140 RPM steps.
- **Over clocked RAM from 2666 to 3200.** Triad 88.6 → 102 GB/s; GLM decode increased about +3–5%. Stability: memtester (as root: as a normal user the memlock limit quietly shrinks it to ~8 MB per copy) plus a 3-hour DeepSeek run, confirming ECC (error corrections) at 0 corrected / 0 uncorrected throughout.
- **`iommu=pt` on the kernel command line.** +3.5% prefill for tensor-parallel llama.cpp, and NVMe random reads recovered (KIOXIA QD32 837K → 941K IOPS).
- **BIOS:** - APBDIS = 1 with fixed SOC P-state P0, which keeps the Infinity Fabric at full clock (about +20 W at idle). - Determinism slider set to Power. - **Resizable BAR on** (it was off: BAR1 was 256 MiB). - PCIe Ten Bit Tag on. Together: another +4% tensor-parallel prefill and a few % on GLM decode.
- **llama.cpp `-sm tensor`: `-ub 1024` instead of 512.** Without P2P, the two cards relay every layer's results through system RAM, across the I/O die. A bigger micro-batch halves the number of relays: +5% on Qwen3.8-27B, +9% on a 27B fine-tune. It needed a little more VRAM, so the split moved slightly toward the 3090.
#AMD EPYC 7443#Supermicro H12SSL-NT#Intel Core Ultra 7 265KF#MSI Z890#NVIDIA RTX 3090#NVIDIA RTX 5070 Ti#NVIDIA RTX 5060 Ti#llama.cpp#GLM-5.3-Flash#DeepSeek-V4.1-Flash#Qwen3.8-27B#Strata