Positron AI Says Inference Now Shapes Infrastructure Deployment
Positron AI executive Cameron McCaskill said inference accounts for about two-thirds of workloads running through AI infrastructure, compared with roughly 85% training and 15% inference in 2023. He said Positron’s Atlas and Titan inference appliances use commodity memory rather than scarce high-bandwidth memory, enabling air-cooled deployments and modular data centers. The workload figures and broader deployment claims were not independently measured in the report.
The interview highlights how vendors are positioning inference-oriented hardware and modular data centers as AI infrastructure priorities, though its market-wide workload estimates come from a single vendor executive.
AI spending is shifting from giant training clusters toward smaller, quickly built sites, and modular data centers are becoming a natural home for inference workloads. Power, cooling and memory supply now shape where tokens get generated as much as raw chip performance does.
Inference-first chipmakers want to capture that shift. Positron AI Inc., which raised $875 million at a $5 billion valuation in September, builds inference appliances around commodity memory instead of scarce high-bandwidth memory. Inference has overtaken training as the force behind infrastructure buying decisions, according to Cameron McCaskill (pictured), head of sales and business development at Positron.
“We’re at a point where about two-thirds of the workloads being run through these AI infrastructures are inference versus one-third is training. And back in 2023, it was like 85% training, 15% inference,” McCaskill said. “So now you can see why inference is driving the economic decisions of how they’re going to build out their infrastructure.”
McCaskill spoke with theCUBE Research’s Rebecca Knight and John Furrier at SC26, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed inference overtaking training, Positron’s commodity-memory chip design and why low-power modular data centers suit inference. (* Disclosure below.)