Ubicloud

Headlines · 1

1. 6d
   vLLM constrained decoding lifts tool-call schema accuracy

   Ubicloud engineers reported that serving Kimi K2.6 with vLLM 0.28 produced valid tool-call schemas in 92% of 2,000 K2VV ToolCall benchmark requests. After enabling constrained decoding in vLLM 0.31, they said schema accuracy reached 100% for newer Kimi and DeepSeek models before deploying the change in production. The post also warns that constrained decoding can introduce off-course generation, grammar-translation errors, request failures, and conflicts with speculative decoding, so vLLM did not enable it by default.

   Hacker News 首页（100+ 分） · 🔥 0

Experience and discussion from the community

Share my Ubicloud experienceAsk about Ubicloud