CNCF探讨利用Kubernetes编排智能体AI推理
Cloud Native Computing Foundation执行总监Jonathan Bryce在theCUBE的一段赞助访谈中表示,随着编程智能体反复发起推理调用,智能体AI正为Kubernetes基础设施带来新的需求。Bryce称,一个编程智能体可生成10个子智能体,每个子智能体都可能在数小时内发起数千次推理调用。CNCF新设的AI推理与智能体专题讨论了相关项目:llm-d推理引擎和Agent Substrate旨在提高现有硬件的利用率,OpenShell则专注于智能体的安全与控制。报道还称,Nvidia捐赠OpenShell后,该项目进入CNCF的中立治理。Linux Foundation赞助了这段访谈,但表示不对内容拥有编辑控制权。
与早期聊天机器人每条消息通常只触发一次推理调用不同,智能体在完成任务时会反复调用模型。报道将模型服务、硬件利用、安全和控制列为这类工作负载的基础设施需求,因此编排成为CNCF智能体AI工作的重点。
The rise of agentic AI systems is giving Kubernetes, long valued for its reliability and predictability, an unexpected second act.
As AI spending shifts from training models toward serving them and running agents, the open-source community is developing tools to handle the growing volume of inference requests. The orchestration platform was built from the start for dynamic, distributed workloads, which gave fast-growing AI companies a reliable foundation, according to Jonathan Bryce (pictured), executive director of the Cloud Native Computing Foundation.
“The AI boom has sparked a massive investment and a huge build-out of infrastructure overall,” Bryce said. “Having Kubernetes built from the beginning to handle really dynamic distributed workloads as those companies were growing like crazy — they had a really reliable, solid foundation to build on at the same time.”
Bryce spoke with theCUBE’s Rebecca Knight at KubeCon + CloudNativeCon NA, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how CNCF projects can meet rising inference demand, make better use of data center hardware and secure AI agents. (* Disclosure below.)
#Cloud Native Computing Foundation#Kubernetes#llm-d#Agent Substrate#OpenShell#Nvidia