vLLM constrained decoding lifts tool-call schema accuracy
Ubicloud engineers reported that serving Kimi K2.6 with vLLM 0.28 produced valid tool-call schemas in 92% of 2,000 K2VV ToolCall benchmark requests. After enabling constrained decoding in vLLM 0.31, they said schema accuracy reached 100% for newer Kimi and DeepSeek models before deploying the change in production. The post also warns that constrained decoding can introduce off-course generation, grammar-translation errors, request failures, and conflicts with speculative decoding, so vLLM did not enable it by default.
Ubicloud’s deployment demonstrates that inference-engine configuration can materially affect tool-call schema adherence in multi-turn agentic workloads, while its documented compatibility issues show why the feature remains opt-in.
All Blog Posts
October 20, 2026 · 8 min read
Jesse Donkervliet and Junhao Li
Senior Software Engineers
When I use an open-weight model, I see results that are nowhere close to what the quality benchmarks indicate.
#vLLM#Ubicloud#Kimi K2.6#DeepSeek V4 Flash#XGrammar#K2VV ToolCall#OpenCode