Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Marvell and Penguin Detail Cross-Rack Photonic Memory Pooling

AI summary

Penguin Solutions' MemoryAI Photonic Appliance places 16TB of HBM and DDR5 in a separate enclosure connected to GPUs through Marvell's Photonic Fabric, allowing servers across multiple racks to share memory. Marvell and Penguin representatives said the approach could reduce data movement and improve AI data-center efficiency, while claiming more than 80% of scale-up traffic consists of memory transactions. These traffic and energy figures come from a sponsored vendor interview and have not been independently validated.

In-depth insight

The appliance provides a concrete shared-memory design for AI inference workloads that need long context or frequent access to data spread across multiple racks.

Memory bottlenecks are emerging as one of the costliest problems in AI infrastructure as inference overtakes training. GPUs sit idle waiting on data, while high-bandwidth memory, or HBM, stays bolted to processors that cannot share it.

Photonics is shaping up as one way out. Marvell Technology Inc. has been pairing optics with photonic memory to stretch memory beyond the edge of a chip. The payoff is better efficiency in data centers enterprises have already paid for, according to Preet Virk (pictured, left), senior vice president and general manager of the Photonic Fabric Business Unit at Marvell.

“In the data center today, we have a lot of room for improvement of efficiency,” Virk said. “And when I say efficiency, I’m alluding to the wasted flops because we have GPU utilization that’s very low.”

Virk and Andy Mills (right), vice president of advanced product development at Penguin Solutions Inc., spoke with theCUBE Research’s John Furrier and Amit Eyal Govrin at SC26, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how photonics and shared memory pools can ease memory bottlenecks in AI infrastructure. (* Disclosure below.)

#Penguin Solutions#Marvell Technology#Penguin MemoryAI Photonic Appliance#Marvell Photonic Fabric#HBM#DDR5

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.