迈威尔与Penguin详解跨机架光子内存池
Penguin Solutions的MemoryAI Photonic Appliance通过Marvell的Photonic Fabric,将16TB的HBM和DDR5置于独立设备中并连接GPU,使多个机架的服务器共享内存。迈威尔与Penguin代表称,该方案可减少数据移动并提升AI数据中心效率,同时声称扩展网络流量中超过80%为内存事务。相关流量与能耗数据来自赞助的厂商访谈,尚未得到独立验证。
该设备为需要长上下文或频繁访问跨机架数据的AI推理负载提供了一种具体的共享内存设计。
Memory bottlenecks are emerging as one of the costliest problems in AI infrastructure as inference overtakes training. GPUs sit idle waiting on data, while high-bandwidth memory, or HBM, stays bolted to processors that cannot share it.
Photonics is shaping up as one way out. Marvell Technology Inc. has been pairing optics with photonic memory to stretch memory beyond the edge of a chip. The payoff is better efficiency in data centers enterprises have already paid for, according to Preet Virk (pictured, left), senior vice president and general manager of the Photonic Fabric Business Unit at Marvell.
“In the data center today, we have a lot of room for improvement of efficiency,” Virk said. “And when I say efficiency, I’m alluding to the wasted flops because we have GPU utilization that’s very low.”
Virk and Andy Mills (right), vice president of advanced product development at Penguin Solutions Inc., spoke with theCUBE Research’s John Furrier and Amit Eyal Govrin at SC26, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how photonics and shared memory pools can ease memory bottlenecks in AI infrastructure. (* Disclosure below.)
#Penguin Solutions#Marvell Technology#Penguin MemoryAI Photonic Appliance#Marvell Photonic Fabric#HBM#DDR5