ScaleFlux bets smarter SSD controllers can break the AI memory bottleneck
ScaleFlux VP of products and marketing Pichai Balaji told theCUBE at the AI Data Pipeline Forum that a GPU runs at roughly 20–22 terabytes per second while DRAM is about 37–40 times slower and storage roughly 800 times slower, so GPUs increasingly sit idle waiting on data as the bottleneck moves from compute to memory and storage. ScaleFlux sells SSD controllers and Compute Express Link memory controllers along with firmware and a full reference architecture, positioning NAND flash as active memory for key-value cache offload in inference rather than as passive archive storage. Balaji said the company has demonstrated more than 200 flexible data placement streams and 7 to 10 drive writes per day on standard triple-level-cell NAND, and that it is NAND-agnostic, offering reference designs and contract-manufacturing support around whatever flash customers can source.
As the bottleneck shifts from compute to memory and storage, treating flash as active memory may be a practical way to keep expensive GPUs from idling.