Lenovo Research

Headlines · 1

1. Oct 23
   Lenovo touts pooled memory plus near-memory computing to ease KV cache strain

   TMTpost reports that a Lenovo team led by chief scientist Xiao Limin has built a compute-storage fusion pooled system combining memory pooling with near-memory computing to address two inference pain points: insufficient GPU and CPU-attached memory for KV cache, and high-cost GPU access to external storage. The team places the system at a "G2.5" stage and describes vendor self-tests. Versus NVMe-tier pooling, memory-pool offloading cut first-token latency by 3.5x at fixed input length and 2.4x in multi-turn dialogue. Cross-instance KV reuse accelerated first-token latency by up to 12–28x versus full recomputation and 2–3x versus an RDMA distributed memory pool. Offloading attention computation to the memory-pool side cut decode latency by 2.5x, and end-to-end inference latency was 2–3x faster than the RDMA pool, with system throughput 6–20x higher as context lengthened. Lenovo plans to move from G2.5 toward G1.5 using scale-up interconnects in supernodes. Xiao said he has not seen a product that brings near-memory computing into a large-capacity memory pool to address both KV cache capacity and access overhead; no independent third-party verification was reported.

   钛媒体 · 🔥 0

Experience and discussion from the community

Share my Lenovo Research experienceAsk about Lenovo Research