Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

CNCF discusses Kubernetes orchestration for agentic AI inference

AI summary

Agentic AI is creating new demand for Kubernetes infrastructure as coding agents make repeated inference calls, Cloud Native Computing Foundation executive director Jonathan Bryce said during a sponsored theCUBE interview. Bryce said a coding agent can spawn 10 sub-agents, each of which may make thousands of inference calls over several hours. CNCF’s new AI inference and agentic track addresses related projects: the llm-d inference engine and Agent Substrate aim to improve the use of existing hardware, while OpenShell focuses on securing and controlling agents. The report also notes that OpenShell entered CNCF neutral governance following Nvidia’s contribution. The Linux Foundation sponsored the interview segment but said it did not control the editorial content.

In-depth insight

Unlike early chatbots that triggered one inference call per message, agents repeatedly call models during a task. The report identifies model serving, hardware utilization, security and control as infrastructure requirements for these workloads, making orchestration a focus of CNCF’s agentic AI work.

The rise of agentic AI systems is giving Kubernetes, long valued for its reliability and predictability, an unexpected second act.

As AI spending shifts from training models toward serving them and running agents, the open-source community is developing tools to handle the growing volume of inference requests. The orchestration platform was built from the start for dynamic, distributed workloads, which gave fast-growing AI companies a reliable foundation, according to Jonathan Bryce (pictured), executive director of the Cloud Native Computing Foundation.

“The AI boom has sparked a massive investment and a huge build-out of infrastructure overall,” Bryce said. “Having Kubernetes built from the beginning to handle really dynamic distributed workloads as those companies were growing like crazy — they had a really reliable, solid foundation to build on at the same time.”

Bryce spoke with theCUBE’s Rebecca Knight at KubeCon + CloudNativeCon NA, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how CNCF projects can meet rising inference demand, make better use of data center hardware and secure AI agents. (* Disclosure below.)

#Cloud Native Computing Foundation#Kubernetes#llm-d#Agent Substrate#OpenShell#Nvidia

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.