Buoyant
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Buoyant launches open-source Kinference for vLLM
Buoyant announced Kinference, an Apache 2.0-licensed open-source control plane for building inference platforms with vLLM and Kubernetes. It offers token rate limiting, tenant usage and cost tracking, inference-specific load balancing, and policy-based routing through an HTTP endpoint compatible with OpenAI and Anthropic clients. In Buoyant's public benchmarks, Kinference reported lower median time-to-first-token and end-to-end latency than standard Kubernetes load balancing on one workload, along with 7.4 times the goodput on a shared-prefix workload and about 3.5 times on a multiturn workload.
vLLM Blog · 🔥 0
Experience and discussion from the community
Share my Buoyant experienceAsk about Buoyant
Nobody has shared their experience with Buoyant yet.