Positron AI称推理正重塑基础设施部署
Positron AI高管Cameron McCaskill称,目前AI基础设施工作负载中约三分之二来自推理,而2023年训练和推理占比分别约为85%和15%。他表示,Atlas和Titan推理设备采用通用内存而非稀缺的高带宽内存,可支持空气冷却和模块化数据中心部署。报道未对这些工作负载比例及更广泛的市场判断进行独立验证。
这次采访展示了厂商如何将推理硬件和模块化数据中心定位为基础设施重点,但市场工作负载估算仅来自单一厂商高管。
AI spending is shifting from giant training clusters toward smaller, quickly built sites, and modular data centers are becoming a natural home for inference workloads. Power, cooling and memory supply now shape where tokens get generated as much as raw chip performance does.
Inference-first chipmakers want to capture that shift. Positron AI Inc., which raised $875 million at a $5 billion valuation in September, builds inference appliances around commodity memory instead of scarce high-bandwidth memory. Inference has overtaken training as the force behind infrastructure buying decisions, according to Cameron McCaskill (pictured), head of sales and business development at Positron.
“We’re at a point where about two-thirds of the workloads being run through these AI infrastructures are inference versus one-third is training. And back in 2023, it was like 85% training, 15% inference,” McCaskill said. “So now you can see why inference is driving the economic decisions of how they’re going to build out their infrastructure.”
McCaskill spoke with theCUBE Research’s Rebecca Knight and John Furrier at SC26, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed inference overtaking training, Positron’s commodity-memory chip design and why low-power modular data centers suit inference. (* Disclosure below.)