Lenovo touts pooled memory plus near-memory computing to ease KV cache strain
TMTpost reports that a Lenovo team led by chief scientist Xiao Limin has built a compute-storage fusion pooled system combining memory pooling with near-memory computing to address two inference pain points: insufficient GPU and CPU-attached memory for KV cache, and high-cost GPU access to external storage. The team places the system at a "G2.5" stage and describes vendor self-tests. Versus NVMe-tier pooling, memory-pool offloading cut first-token latency by 3.5x at fixed input length and 2.4x in multi-turn dialogue. Cross-instance KV reuse accelerated first-token latency by up to 12–28x versus full recomputation and 2–3x versus an RDMA distributed memory pool. Offloading attention computation to the memory-pool side cut decode latency by 2.5x, and end-to-end inference latency was 2–3x faster than the RDMA pool, with system throughput 6–20x higher as context lengthened. Lenovo plans to move from G2.5 toward G1.5 using scale-up interconnects in supernodes. Xiao said he has not seen a product that brings near-memory computing into a large-capacity memory pool to address both KV cache capacity and access overhead; no independent third-party verification was reported.
The report frames KV cache placement as a central bottleneck as AI applications shift from single-turn chatbots to multi-turn agents, citing CAICT data that China's average daily token consumption rose from 100 billion in 2024 to 500 trillion by August 2025.

过去二十年,服务器峰值浮点算力每两年增长约3倍,而DRAM带宽每两年只增长1.6倍,互连带宽仅1.4倍,这就形成了行业内称之为存储墙的痛点,而这个问题其实已经存在很久,而如今的AI时代,随着计算能力的增强及其对数据需求量的扩大,据中国信通院统计,中国每天平均的Token消耗量,从2024年的1000亿,跃升到2025年8月的500万亿,一年多时间增长了约5000倍。在此背景下,存储墙问题也被放大了。
推理时代,算力与存储必须加强融合
大模型推理的核心流程之一,是自回归解码,模型每生成一个Token,都要回头看一遍之前所有的上下文。如果每次都重新计算全部历史数据,算力会被海量的重复计算消耗殆尽。于是KV Cache诞生了,它把中间计算结果缓存起来,用“以存换算”的方式跳过重复计算。这个机制极大地减少了计算开销,但它同时埋下了一颗雷:数据越攒越多,而缓存总得有地方放。
#Lenovo#Lenovo Research#China Academy of Information and Communications Technology#NVMe#CXL#RDMA