Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Baseten Adds Deferred Tool Loading for Open Models

AI summary

Baseten says its Model APIs now support deferred tool loading for DeepSeek, Kimi, and GLM in the Chat Completions and Messages API formats. In its benchmarks using Claude Code, coding tasks were on average 62.5% cheaper and 26.2% faster, while agentic tasks were 51.1% cheaper and 14.9% faster. The tests used 55 tasks across eight benchmarks. Baseten reported that DeepSWE and MCP-Atlas tasks were slightly slower, while eager loading was more efficient on Toolathlon-GYM, where tasks averaged 68 tool calls.

In-depth insight

Baseten says deferred tool loading can reduce context and serving costs while keeping the same model and harness. In its example, tool schemas in the system prompt fell from about 141,000 tokens to about 1,300 tokens, which the company says can make decoding cheaper and faster.

Deferred tool loading: Make your agent 62.5% cheaper without changing your model or harness

Please rotate your device

To continue exploring this website, please rotate your device to a vertical orientation

AI engineering

Deferred tool loading: Make your agent 62.5% cheaper without changing your model or harness

We brought deferred tool loading to open models, making coding agents up to 62% cheaper and 26% faster with the exact same model and harness.

#Baseten#Baseten Model APIs#DeepSeek#Kimi#GLM#GLM 5.3#Claude Code#Codex

Original · 6 min readRead the original
Group chat

No comments yet. Start the conversation.