Marvell and Penguin Detail Cross-Rack Photonic Memory Pooling
Penguin Solutions' MemoryAI Photonic Appliance places 16TB of HBM and DDR5 in a separate enclosure connected to GPUs through Marvell's Photonic Fabric, allowing servers across multiple racks to share memory. Marvell and Penguin representatives said the approach could reduce data movement and improve AI data-center efficiency, while claiming more than 80% of scale-up traffic consists of memory transactions. These traffic and energy figures come from a sponsored vendor interview and have not been independently validated.
The appliance provides a concrete shared-memory design for AI inference workloads that need long context or frequent access to data spread across multiple racks.
Memory bottlenecks are emerging as one of the costliest problems in AI infrastructure as inference overtakes training. GPUs sit idle waiting on data, while high-bandwidth memory, or HBM, stays bolted to processors that cannot share it.
Photonics is shaping up as one way out. Marvell Technology Inc. has been pairing optics with photonic memory to stretch memory beyond the edge of a chip. The payoff is better efficiency in data centers enterprises have already paid for, according to Preet Virk (pictured, left), senior vice president and general manager of the Photonic Fabric Business Unit at Marvell.
“In the data center today, we have a lot of room for improvement of efficiency,” Virk said. “And when I say efficiency, I’m alluding to the wasted flops because we have GPU utilization that’s very low.”
Virk and Andy Mills (right), vice president of advanced product development at Penguin Solutions Inc., spoke with theCUBE Research’s John Furrier and Amit Eyal Govrin at SC26, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how photonics and shared memory pools can ease memory bottlenecks in AI infrastructure. (* Disclosure below.)
#Penguin Solutions#Marvell Technology#Penguin MemoryAI Photonic Appliance#Marvell Photonic Fabric#HBM#DDR5